2 Replies Latest reply on Aug 29, 2011 7:59 AM by pmuir

    Invalidation mode cache in a cluster with singleton store

    ydewit
    ydewit Aug 16, 2011 9:47 PM

      I am a bit confused by the behavior I am seeing when using Infinispan in invalidation mode.

       

      I have a simple Junit that creates two DefaultCacheManager's (CM1 and CM2) to simulate a cluster of two nodes. Each DefaultCacheManager has a single cache configured with INVALIDATION/SYNCHRONOUS mode and a custom FileCacheStore in singleton mode that can read/write from/to the filesystem. CM1 is the coordinator.

       

      When I put an entry in CM1->DATA cache, the FileCacheStore is called and the entry is stored in the file system. I then read the same entry from CM2-DATA and I see CM2's FileCacheStore being called to read the entry directly from the filesystem. I am assuming that entry locking across the cluster will make sure that I can read the filesystem from CM2 without having CM1 write to the same entry at the same time.

       

      Now, when I put an entry in CM2->DATA, CM2's FileCacheStore is not called (I guess it is not active since the coordinator is still CM1, so fine), but I also don't see the entry being forwarded to CM1 so it can be written to the filesystem by CM1, the coordinator. The test fails since the entry is not in the filesystem.

       

      Why is that? Shouldn't CM2 forward copy the new entry to CM1 to have it store in the filesystem?

        • 1. Re: Invalidation mode cache in a cluster with singleton store
          ydewit
          ydewit Aug 18, 2011 12:42 PM (in response to ydewit)

          I looked again at the documentation but couldn't find a description of what is the expected behavior when a cache is configured with INVALIDATION/SYNC, shared=false and using a SingletonStore. Any pointers to what is the as-designed case?

           

          I also changed my cache configuration: I removed the SingletonStore and made the loaders shared. I now see the cache working properly in my test cases and saw that each node is now directly reading and writing to the CacheStore. The assumption I need to validate now is whether the locks are distributed (it must be, but need to make sure) so that consistency is guaranteed in a cluster.

           

          For a little background, I am using Infinispan to cluster a filesystem that could be Gbytes in size. I will have a REPLICATION cache with only metadata for the entire filesystem and a INVALIDATION/shared cache for the actual file contents with a sensible eviction policy.

            • Actions
          • 2. Re: Invalidation mode cache in a cluster with singleton store
            pmuir
            pmuir Aug 29, 2011 7:59 AM (in response to ydewit)

            In invalidation mode, Infinispan passes all responsibility for replicating data to the cache store, and doesn't replicate the data at all. So in your scenario, if CM2 receives a write, it needs to write this to the cache store (in your case, the filesystem) itself. It won't pass the entry to CM1 for writing. The coordiantor shouldn't matter.

             

            The docs on shared cache stores's aren't clear, I agree, and I've added it to the list of stuff to improve in the docs.

             

            When a cache store is marked as shared, it means that the same cache store is present on every node, so only the originating node will store the data. So I'm surprised that in your first case that CM2 didn't write through the key to the cache store. Can you debug into infinispan to see why?

              • Actions