Apparently, AWS does not support UDP multicast between machine instances (even those in the same security group). So this, I believe, rules out the discovery group approach to HornetQ clustering. AWS instances get IP addresses assigned from a very large pool and cannot be made static. So this, I believe, rules out specifying members of a cluster explicitly.
Has anyone done any research or documentation on implementing a HornetQ cluster in the Cloud (AWS to be specific)?
IIRC The way JGroups/Infinispan solved this problem was by having a filesystem based "discovery" mechanism.
Assuming each node has access to a shared file system - each node writes some information on it's address to a shared file(s) in a well known location when it starts up, other nodes then read that file(s) to "discover" what other nodes are in the cluster.
We could do something similar in HQ. You could add a feature request if you like.
JGroups has a few options regarding discovery on AWS:
- GossipRouter is an external lookup process (you can have multiple of those to prevent SPOFs), which is used for discovery only. After initial discovery, nodes talk to each other directly
- FILE_PING uses the file system to store and read information about cluster nodes. Note that a shared file system, like a Samba /CIFS or NFS mount is needed, unless you run all of your instances on the same local box
- S3: you can use an S3 bucket to do discovery (S3_PING)
This code is pretty simple, and can be copied from JGroups