I just want to confirm something that I'm observing, and to ask some questions about it. Here's the observation:
Let's say I have a single cache on a cluster with two nodes A and B. Assume the cache starts completely empty. Given this:
- Node A reads X from some data store, then calls cache.put(X), pushing some distinct object X into the cache
- Node B then also reads the same X from some data store (A's X equals() B's X), then calls cache.put(X).
- Node A's copy of X is invalidated.
In other words, X can exist in one and only one node in the cluster in that cache.
- Is this expected behavior?
- Is there any way to turn it off?
- If not, then it seems that the only way to have multiple nodes in a cluster to have X in their caches is to use replication?
This is an important question for us, because it goes to the heart of whether we can use Infinispan to cache our heavy-read-light-write tables, in which all rows of the table are typically cached in memory because they change so infrequently.
Hi Darrell, with regards to your questions:
Is this expected behavior?
It's a little bit different than what you described. A key/value pair can exist in more than one node. The invalidation mode is optimized for read operations so if you read X subsequently from all nodes, then X will be stored in all nodes but only until you do a write on some of the nodes. After a write operation, X is invalidated in all other nodes but once the other nodes read the X, they again hold the value.
Is there any way to turn it off?
AFAIK no, this is expected bahavior of the invalidation mode.
If not, then it seems that the only way to have multiple nodes in a cluster to have X in their caches is to use replication?
If you really do not want to use invalidation, and not even replication mode where keys are stored on all nodes, you can try a distribution mode with L1 cache enabled.
I found the answer. The trick is to use cache.putForExternalRead() not put(). This causes entries to be put into cache without sending out invalidation messages to the rest of the cluster, which is exactly the way our ORM wants to use the cache. So now we can use invalidation caches for tables that change infrequently but are referred to heavily.