6 Replies Latest reply on May 11, 2012 2:50 AM by mgencur

    cluster wide lock on a whole cache or make it readonly

    dex80526

      Hi all: I have the following use case:

       

      I  have 2 name caches (cache1 and cache2)  and configured in REPLICATION (Async) mode in a cluster. 

       

      There are situations which require to not allow modifications to cache2 from any other nodes when a node is modifying cache1.

      In other words, I need to lock the whole cache across cluster.

       

      Here is a 2 node cluster situation.

       

      I need a way to allow me to do the following (in sudo code):

       

      on node1:

       

      acquire-cluster-wide-lock(cache2);

      modify cache items in cache1; //for cache1, I am using the lazy lock.

      .....

      modify cache items in cache2;

      ....

      release-cluster-wide-lock(cache2);

       

       

      when the cluster-wide-lock is acquired by node1, node2 can not modify to cache2.

       

      Ideally, nodes still be able to read the caches from cache2.

       

      Is there a way to implement this in ISPN?

        • 1. Re: cluster wide lock on a whole cache or make it readonly
          dex80526

          There is no response so far.

           

          Let me add some extra explaination to the use case:

           

          1) I was store cipher key in cache1, which is used to encrypt/decrypt the data store in cache 2.

           

          2) I have a schedule the task to rotate/change the cipher key, which will create a new cipher key and re-cncrypt data in cache 2.

            This action occures only on the coordinator. After a sucess roration, I'll remove the old cipher key in cahe1.

           

          Here is the problem:

          there is no way to gurantee that during the cipher key rotation on the coordinator (say node1), node 2 may add new entries to cache 2 encrypted with the old cipher key, since the current cache lock does not prevent other nodes from reading the old value.

           

          In my view, to guranteen the cluster-wide transactional operation, I need to be able to lock the cache2  (i.e., does not allow any other node to modify the state).

           

          After reading some document, one option I came up is to have another cache, say, cluster-lock, which will have only one constant entry to serve as the "lock-token" across the cluster.  Before every modification operation to cache2, the code has to first lock the "lock-token".

           

          Any suggestions and comments.

          • 2. Re: cluster wide lock on a whole cache or make it readonly
            mgencur

            The data being encrypted does not have to occupy whole cache. It could be probably stored as one cache entry and then you can ensure locking on that key/value pair. In one transaction, first compute a new cipher, get the old data and store newly encrypted data, and store new cipher inside the same transaction.

             

            With pessimistic locking in use, the put() operation will ensure lock acquisition and other users won't be able to modify the cipher/data until the end of the transaction.

            • 3. Re: cluster wide lock on a whole cache or make it readonly
              dex80526

              yes, it could work just using pessmic locking on the cipher key. But this requires to aquire lock for reading as well.  That will cause significant performance issue.

              • 4. Re: cluster wide lock on a whole cache or make it readonly
                mgencur

                No, read operations are non-blocking. See Infinispan documentation - Locking and Concurrency.

                • 5. Re: cluster wide lock on a whole cache or make it readonly
                  dex80526

                  It will not work if we do not aquire lock on reading. Here is why (see below);

                   

                  Node 1:                      Node2:

                   

                  (c1, cipher1),  (k1, v1)                   (c1, cipher1),   (k1, v1)

                  (c1, cipher1),  (k2, v2)                    (c1, cipher1),   (k2, v2)

                   

                  tx.begin()                               adding a new entry with latest (existing cipher key = get(c1))

                  lock(cipher1)                          (c1, cipher1), (k3, v3)

                  make new cipher

                  reencrypt data

                  cipher2, (k1, v1')

                  cipher2, (k1, v2')

                  remove (cipher1)

                  unlock()

                  tx.commit()

                   

                  After the transaction, the caches look like this:

                  (c2, cipher2), (k1,v1') (k1,v2')  (k3, v3)

                   

                  Here, v1' and v2' are now encrypted by cipher2, but v3 is encrypted by cipher1.

                   

                  At this point, the cipher1 is removed, now the (k3, v3) is never able to be decrypted any more. The reason is that it is encrypted with cipher1 which is removed by the node1.  You can see the cipher1 is still accessible during the transaction on node1.  That's my issue.

                  • 6. Re: cluster wide lock on a whole cache or make it readonly
                    mgencur

                    I see, then something like the "lock-token" could help, as you described above.