-
1. Re: Synchronization between clusters
galder.zamarreno Aug 23, 2012 2:27 AM (in response to pedrokowalski)1 of 1 people found this helpfulWe're working on a global solution as part of https://community.jboss.org/docs/DOC-17546, but the 1st alpha of this won't be ready for a few weeks.
There's some hacky workarounds you could use, but I'd wait for the alpha release which should be coming soon...
The JIRA to monitor is: https://issues.jboss.org/browse/ISPN-1824
-
2. Re: Synchronization between clusters
pedrokowalski Aug 23, 2012 3:41 AM (in response to galder.zamarreno)Galder, thanks for your reply. Good to know that in the nearest future this will be easily achievable :-)
Is it safe to use the RELAY protocol for Infinispan replication cluster mode?
I've tried to achieve something similar to the bridge concept but rather on a naive level: cluster {A, B, C} using UDP transport and cluster {C, D, E} using TCP transport. The C is a node of the both clusters - it listens for changes in cluster {A, B, C} and puts the changes in cluster {C, D, E}. However, I don't think that it'll work if the {A, B, C} and {C, D, E} should be in different networks, right?
When it comes to the transactions, there is following excerpt in the page you showed: "in the case of the transactions, it would be the commit message that triggers the sending of transaction information to the remote site". So if a transaction in cluster C1 has commited only then I send changes to the cluster C2. What happens if the C2 fails to add the entries to the cache? Does the 'commit message' referrs to the 1PC or the prepare-commit of 2PC?
I'm just asking because I would like to be sure that any changes made to cache in Cluster C1 will be consistent with Cluster C2. If the transaction will be rolled back within the C1, the C2 won't be even notified by it and this seems a simple case. But what when a transaction in C1 is NOT rolled back (it commits successfully) but for whatever reason the cache changes cannot be made and replicated to Cluster C2. Then, the global transaction should be rolled back (neither C1 nor C2 has the entries modified by the transaction). Will this case be also supported?
Thanks in advance!
-
3. Re: Synchronization between clusters
mircea.markus Aug 24, 2012 8:35 AM (in response to pedrokowalski)1 of 1 people found this helpfulIs it safe to use the RELAY protocol for Infinispan replication cluster mode?
It is technically named RELAY2 (relay was the first version of this protocol used as POC). It will work with replicated caches, yes. It is actually possible to have a replicated cache in site A which backups to a distributed cache in site B.
I've tried to achieve something similar to the bridge concept but rather on a naive level: cluster {A, B, C} using UDP transport and cluster {C, D, E} using TCP transport. The C is a node of the both clusters - it listens for changes in cluster {A, B, C} and puts the changes in cluster {C, D, E}. However, I don't think that it'll work if the {A, B, C} and {C, D, E} should be in different networks, right?
it will work with {A,B,C} in one network/location (e.g. LON) and {C,D,E} in a different location (e.g. NYC). The updated design document for x-site replication can be found here: https://community.jboss.org/wiki/DesignForCrossSiteReplication
When it comes to the transactions, there is following excerpt in the page you showed: "in the case of the transactions, it would be the commit message that triggers the sending of transaction information to the remote site". So if a transaction in cluster C1 has commited only then I send changes to the cluster C2. What happens if the C2 fails to add the entries to the cache? Does the 'commit message' referrs to the 1PC or the prepare-commit of 2PC?
The given document was revisited and the final design is here: https://community.jboss.org/wiki/DesignForCrossSiteReplication.
You'd be basically allowed to have the 2nd cluster be a full tx participant, i.e. answer to both prepare and commit. There's no mutch point in doing x-site 2PC if the 2nd cluster is passive: there won't be any writes going in it and the prepare phase(i.e. when locks are acquired) must succeed.
Re: what happens if the changes cannot be committed to the 2nd cluster: that would be configurable - we might fail the transaction or allow it to succed. If later, then the admin of the sites must reconciliate the state.
I'm just asking because I would like to be sure that any changes made to cache in Cluster C1 will be consistent with Cluster C2. If the transaction will be rolled back within the C1, the C2 won't be even notified by it and this seems a simple case. But what when a transaction in C1 is NOT rolled back (it commits successfully) but for whatever reason the cache changes cannot be made and replicated to Cluster C2. Then, the global transaction should be rolled back (neither C1 nor C2 has the entries modified by the transaction). Will this case be also supported?
yes, (se above) this will be configurable.
May I suggest to give a read to nbew design document[1] and let us know any concerns you might have.
[1] https://community.jboss.org/wiki/DesignForCrossSiteReplication.
-
4. Re: Synchronization between clusters
pedrokowalski Aug 27, 2012 6:18 AM (in response to mircea.markus)Thanks a lot Mircea - things are clearer right now.
To explain why I want to achieve such thing is that I have the following constraints:
- nodes within the cluster must use UDP for transport,
- the JTA/XA transaction support must exist between clients and the cluster,
- some clients are not allowed to use UDP and must therefore fallback to TCP instead.
So, given such constraints I was thinking about using the Hot Rod protocol. Unfortunately (https://community.jboss.org/message/754516#754516) XA transactions doesn't work with Hot Rod. Ok, so the next idea was to treat client as a cache node (which gives me the JTA/XA support) and use the TCP bridge between the client-node and the cluster (which can still run in UDP).
And thit is the moment I think I'm overcomplicating things here... I mean - are my constraints really so uncommon (the support for Tx and clients that cannot use the UDP transport)?
What do the others do in such case? Do they revert to the TCP cluster and sacrifice stability and performance? Am I missing something or do I have to wait for the next Infinispan/JGroups releases to solve this problem? (I'm currently also required to use Infinispan 5.1.5)
Thanks once again!
-
5. Re: Synchronization between clusters
belaban Oct 24, 2012 2:36 AM (in response to pedrokowalski)Piotr,
you may want to take a look at RAC (https://community.jboss.org/wiki/RAC): it provides almost the same guarantees as transactional replication across sites, but is much faster, especially if you have high latency on the WAN link connecting the sites.
-
6. Re: Synchronization between clusters
belaban Oct 24, 2012 2:37 AM (in response to belaban)Note that RAC is the working title only, I'm trying to find a better name...
-
7. Re: Synchronization between clusters
pedrokowalski Oct 24, 2012 4:30 AM (in response to belaban)Thanks Bela!
I will take a look at it.
BTW. The name recalls me of Oracle RAC ;-)
-
8. Re: Synchronization between clusters
belaban Oct 24, 2012 5:34 AM (in response to pedrokowalski)That's why I'm looking for another name...
-
9. Re: Synchronization between clusters
moia Oct 26, 2012 10:02 AM (in response to pedrokowalski)Have you tried RemoteCacheStore?
I've been looking through the docs recently and this seems like something you want to look into.
Best regards,
Mikolaj
-
10. Re: Synchronization between clusters
pedrokowalski Oct 29, 2012 5:41 AM (in response to moia)Thanks Mikołaj for your input.
As far as I can see from the Javadoc (http://docs.jboss.org/infinispan/4.1/apidocs/org/infinispan/loaders/remote/RemoteCacheStore.html) it seems that RCS works with HotRod. This means that I can only use the TCP connection, while one of the requirements is to use the UDP transport protocol as well, so I'm not sure if it's achievable in this way.
Cheers!
-
11. Re: Synchronization between clusters
moia Oct 30, 2012 5:44 AM (in response to pedrokowalski)Indeed, I missed the requirement, that you need to communicate between clusters with both TCP and UDP.