8 Replies Latest reply: Feb 7, 2011 4:53 AM by vinay a RSS

How to deploy infinispan cluster across several servers????

liqun luo Newbie

Dear every one,

 

         Currently , I want to deploy infinispan cluster across serveral servers, but the tutorial is not clear and detailed? Is any one can write a detailed tutorial about configure infinispan across serveral servers??  Thank you very very much!!!!!

 

Luo

  • 1. Re: How to deploy infinispan cluster across several servers????
    Manik Surtani Master

    Doesn't http://community.jboss.org/wiki/settingupaninfinispancluster work?  You just need to start up instances on > 1 server and they should discover each other.

  • 2. Re: How to deploy infinispan cluster across several servers????
    Savin Seven Newbie

    Dear Manik\All,

     

    Clustering different instances with same Server in Single Machine:

    I was using hot rod server. Within one server, If I start different instances using the below command with different port numbers, Its working fine and detecting each other.

     

    startServer.bat -l 10.38.54.210 -p 6904 -m 20 -t 20 -c ../etc/config-samples/minimal.xml -r hotrod -i 600 -n true -s 1024000 -e 1024000 -o 10.38.54.210 -x 6904
    startServer.bat -l 10.38.54.210 -p 6905 -m 20 -t 20 -c ../etc/config-samples/minimal.xml -r hotrod -i 600 -n true -s 1024000 -e 1024000 -o 10.38.54.210 -x 6905

    Log Info:

    Start1

    Start2

    Start3

    2010-08-16 13:45:36,037 INFO [HotRodOperation] (main) New topology: {/10.38.54.210:6905=5357, /10.38.54.210:6904=8872}

    2010-08-16 13:45:36,037 INFO [TcpTransportFactory] (main) New server added(/10.38.54.210:6905), adding to the pool.

    2010-08-16 13:45:36,037 INFO [TcpTransportFactory] (main) New server added(/10.38.54.210:6904), adding to the pool.

    Start4

    Able to form clustering with different instances of same server.

    Hotrod properties:
    infinispan.client.hotrod.server_list = 10.38.54.210:6905

     

    Clustering different Server in different Machine: Log Info:

    Whereas If I start one more server in different machine[ip -> 10.38.54.192 and port 6900] on the same network with below command, Its failing to detect the already running server instances in the other machine.[above mentioned servers running on port 6904 & 6905 running in ip 10.38.54.210]

     

    startServer.bat -l 10.38.54.192 -p 6900 -m 20 -t 20 -c ../etc/config-samples/minimal.xml -r hotrod -i 600 -n true -s 1024000 -e 1024000 -o 10.38.54.192 -x 6900

     

    Log Info:

    2010-08-16 13:53:24,802 INFO [TestCache] (main) Inside TestCache

    Start1

    Start2

    Start3

    2010-08-16 13:53:24,880 INFO [HotRodOperation] (main) New topology: {/10.38.54.192:6900=7045}

    2010-08-16 13:53:24,880 INFO [TcpTransportFactory] (main) New server added(/10.38.54.192:6900), adding to the pool.

    2010-08-16 13:53:24,880 INFO [TcpTransportFactory] (main) Server not in cluster anymore(/10.38.54.210:6905), removing from the pool.

    Start4

     

    Its giving the info like server (/10.38.54.210:6905) not in cluster and its removig it from the pool but the server was started and running in the another machine but its failing to detect it.

    Hotrod properties:
    infinispan.client.hotrod.server_list = 10.38.54.210:6905;10.38.54.192:6900

    Attached are the files that I have used for clustering. Please let me know, Is there I have to change any configurations in the attached files to achieve clustering hotrod server in different machines. So that it can detect the server that is running in another machine.

     

    My advance thanks for the same.

     

    Thanks & Regards,

    Savin

     

  • 3. Re: How to deploy infinispan cluster across several servers????
    Galder Zamarreño Master

    Your issue does not appear to be related to Hot Rod. You seem to have communications issues between these machines. UDP communication should work between them. You should start testing this http://www.jgroups.org/manual/html/ch02.html#ItDoesntWork, check firewalls...etc.

  • 4. Re: How to deploy infinispan cluster across several servers????
    Savin Seven Newbie

    Dear Galder,

     

    As you said, there seems to be a communication issues between those machines. Thanks for the intimation. Some ports in those machines are closed. When we resolve that issue, it’s started working fine.

     

    As part of understanding, how it’s forming a cluster. I have two questions. If time permits, kindly clarify me.

     

    Machine 1: Server is started using below command

    startServer.bat -l 192.168.1.7 -p 6904 -m 20 -t 20 -c ../etc/config-samples/minimal.xml -r hotrod -i 600 -n true -s 1024000 -e 1024000 -o 192.168.1.7 -x 6904

     

    Machine 2: Server is started using below command

    startServer.bat -l 192.168.1.14 -p 6905 -m 20 -t 20 -c ../etc/config-samples/minimal.xml -r hotrod -i 600 -n true -s 1024000 -e 1024000 -o 192.168.1.14 -x 6905

     

    Question1: [How hotrod Servers detecting each other and forming a Cluster?]

    When you infer both these above commands, we are not mentioning the other machine identity to which hotrod server has to form a cluster. As per my understanding, since both the servers using the same minimal.xml, which in turn uses same jgroups-udp.xml. They form a cluster based on the same mcast_addr and same mcast_port mentioned in the jgroups-udp.xml. Please let me know, whether my understanding is correct. If I miss understand something, how the hotrod servers detecting each other. Kindly clarify it.

     

    Question2: [Each time you start the server, Hotrod servers are connecting to dynamic port of the other machine and forming a cluster. Can we able to fix the ports that the Hotrod server has to connect, instead of connecting to a dynamic port?]

     

    In the first machine, I have used 'netstat -ano' command. I got the below result

    Proto  Local Address              Foreign Address        State                  PID

    TCP    192.168.1.7:3155       192.168.1.14:4679      ESTABLISHED     3852

     

    In the second machine, I have used 'netstat -ano' command. I got the below result

    Proto  Local Address              Foreign Address        State                  PID

    TCP    192.168.1.14:4679      192.168.1.7:3155       ESTABLISHED     11080

     

    From the above two results, we can able to see that they connect to dynamic ports [3155 & 4679], which are not specified anywhere by us. Why this is happening. Is there a way I can able to fix those connecting ports or Will the ports be always decided by Jgroups Transport ? Kindly clarify.

     

    My advance thanks for the same.

     

    Thanks & Regards,

    Savin

  • 5. Re: How to deploy infinispan cluster across several servers????
    Galder Zamarreño Master

    Re Q1: Correct. They see each other thanks to messages they sent to a particular mcast addr+port.

     

    Re Q2: Most likely, that TCP connection is the one belonging to JGroups' FD_SOCK protocol. Set start_port as per: http://community.jboss.org/wiki/JGroupsFDSOCK

  • 6. How to deploy infinispan cluster across several servers????
    vinay a Newbie

    Hi,

     

    The example of GUIDemo of Infinispan shows data accessibility among  inter clusters, created on starting/running multiple instances. So which mode of Clustering this example uses? Is it Distributed mode/Local/mode.. ? Also what technique is used in maintaining the cache here?

     

    Thanks n Regards,

    Vinay

  • 8. How to deploy infinispan cluster across several servers????
    vinay a Newbie

    For the example of GUIDemo of Infinispan, what technique is used in maintaining the cache across all clusters..how is it working?

    How is the caching happening here?