2 Replies Latest reply: Oct 27, 2011 6:02 PM by Scott Stark RSS

Perhaps a dumb S3 question

Scott Stark Master

Would it be possible to run Infinispan in an EC2 environment using only S3 as the store and communication mechanism to share data between jbossas7 instances? The use case for this is in the openshift express environment where the lack of IP6 addresses between nodes that are running multiple express applications does not allow for even TCP based jgroups clustering.

  • 1. Re: Perhaps a dumb S3 question
    Sanne Grinovero Master

    Hi Scott,

    I was going to anwer "no", but actually if some limitations are acceptable we might get something working.

     

    S3 can be used as a cache store, so that each node writes through the cache and stores all data on S3 as well. Then you have to face two other problems:

    • the nodes won't be aware of each other: no locking, hence races on writes have to be acceptable
    • nodes reading data will always read their local copy first, if a key is found in the local cache they won't look in S3, hence you could have stale values, but this might be solved with a relatively short eviction (at the cost of performance).

     

    In pratice I don't know if this could be acceptable for your use case, it certainly would be great to store Lucene indexes as each key value is immutable.

     

    An alternative implementation which would require coding is to implement a different Transport; Infinispan was designed around JGroups but the Transport is in theory pluggable. I write in theory as that wasn't attempted yet, and some of the concepts from JGroups leak out to the API which should be implemented by such an alternative.

  • 2. Re: Perhaps a dumb S3 question
    Scott Stark Master

    That might be ok because the usage is for a simple horizontal application scaling, so as long as the load balancer segments by sessions the lack of locking should be ok. I will take a look at the Transport api to see how that might be leveraged.

     

    Thanks