I have a use-case where I need to
1. Have a big cache in memory which wont hit in one heap, so need it distributed across nodes. What I notice is that every node that comes up reads the entire data from the cacheLoader (so the DB will be hit everytime a new node comes up if I use DB). This seems a bit wasteful since the node that is up could just send all the data that get consistenHashed for the new node. To be more clear about this, imagine node A comes up and reads from the cacheLoader all the 100 keys. When node B comes up it also reads them all from the cacheLoader. Instead node A could send 50 keys to node B and remove the ones it sent. Otherwise the DB is hit pretty hard everytime and that's wasteful. Is this possible, how ?
In my test I used a FileCacheStore
In node A the config was
In node B the config was
Node B still reads from the FileCacheStore.
2. I guess it would be even more efficient to have node A and B both come up parallely and read 50% of data from the DB (CacheStore) each. But how would each node know what data to read from the FileStore such that it would be correct per the CH ?
3. In use-case 1. above if instead of distributed mode I wanted to run node A and B in replicated mode, is it possible? I do want to configure the cacheLoader on both nodes since if only any one cache comes up it should be able to load the data.
May be there is an obvious, simply solution to these use-cases, but it was'nt clear to me from the wikis and my tests with it did'nt work. I'm hoping someone can help me here.
Thanks in advance.
Re 1. I think fetchPersistentState is irrelevant here cos all it does is take another nodes persistent state and apply it to its own cache store. Now, if the cache stores are shared, this is pointless. I'm not sure why B is hitting the FCS, you might wanna check TRACE logs to find out. Maybe it's due to preload being on? Or maybe due to requests hitting the cache and that data not being in memory.
Re 2. That's currently not possible. But you can enable preloading so that both nodes bring in enough data into memory. You can limit how much comes into memory by setting eviction.maxEntries to a particular number. The code that's preloading the cache will only preload up to that number of entries into the cache.
Re 3. Preloading is done on a local fashion, so you'd need both nodes with preloading on.