6 Replies Latest reply on Aug 17, 2015 5:07 AM by sebastian.laskawiec

    How i can to connect two or more JDG nodes (cluster mode) in differents machines??

    andresperez0828

      Hello everybody,

       

      I'm using a Jboss Data Grid 6.5 and i have a cluster with 2 nodes, but this are in the same machine, and need to add a new node, but this node is in other machine, what is the configuration in de Jboss Data Grid for solve this. I have seen in the clustered.xml and i haven't seen nothing respectively.

       

      I have seen something related with Jgroups but i don't know configure it. 

        • 1. Re: How i can to connect two or more JDG nodes (cluster mode) in differents machines??
          sebastian.laskawiec

          Hey Andreas!

           

          Yes that's correct. In $JDG_HOME/bin directory there is a script called clustered.sh. This script will run JDG server in clustered mode using $JDG_HOME/standalone/configuration/clustered.xml.

           

          JGroups is a clustering framework which is used in JDG. I would suggest trying out the defaults and then, depending on your needs, tuning it.

           

          Here are some related topic in the documentation:

          • 2. Re: How i can to connect two or more JDG nodes (cluster mode) in differents machines??
            andresperez0828

            Hi Sebastian,

             

             

            Thanks for your answer,I understand how the cluster mode works, in fact I have a cluster with two nodes in the same machine, each node was started with clustered.sh, however,when I start other node in a different machine this the node could not be connected with the others nodes. I have Jgroups configuration by default in the clustered.xml but nodes or clusters in different computers can´t connect,

             

            best regards,

            • 3. Re: How i can to connect two or more JDG nodes (cluster mode) in differents machines??
              andresperez0828

              this is my clustered.xml, i'm using this file for the nodes  in differents machines.

              -----------------------------------------

              <?xml version='1.0' encoding='UTF-8'?>

               

               

              <server xmlns="urn:jboss:domain:1.6">

                  <extensions>

                      <extension module="org.infinispan.server.endpoint"/>

                      <extension module="org.jboss.as.clustering.infinispan"/>

                      <extension module="org.jboss.as.clustering.jgroups"/>

                      <extension module="org.jboss.as.connector"/>

                      <extension module="org.jboss.as.deployment-scanner"/>

                      <extension module="org.jboss.as.jdr"/>

                      <extension module="org.jboss.as.jmx"/>

                      <extension module="org.jboss.as.logging"/>

                      <extension module="org.jboss.as.modcluster"/>

                      <extension module="org.jboss.as.naming"/>

                      <extension module="org.jboss.as.remoting"/>

                      <extension module="org.jboss.as.security"/>

                      <extension module="org.jboss.as.threads"/>

                      <extension module="org.jboss.as.transactions"/>

                      <extension module="org.jboss.as.web"/>

                  </extensions>

                  <management>

                      <security-realms>

                          <security-realm name="ManagementRealm">

                              <authentication>

                                  <local default-user="$local"/>

                                  <properties path="mgmt-users.properties" relative-to="jboss.server.config.dir"/>

                              </authentication>

                          </security-realm>

                          <security-realm name="ApplicationRealm">

                              <authentication>

                                  <local default-user="$local" allowed-users="*"/>

                                  <properties path="application-users.properties" relative-to="jboss.server.config.dir"/>

                              </authentication>

                              <authorization>

                                  <properties path="application-roles.properties" relative-to="jboss.server.config.dir"/>

                              </authorization>

                          </security-realm>

                      </security-realms>

                      <management-interfaces>

                          <native-interface security-realm="ManagementRealm">

                              <socket-binding native="management-native"/>

                          </native-interface>

                          <http-interface security-realm="ManagementRealm">

                              <socket-binding http="management-http"/>

                          </http-interface>

                      </management-interfaces>

                  </management>

                  <profile>

                      <subsystem xmlns="urn:jboss:domain:logging:1.4">

                          <console-handler name="CONSOLE">

                              <level name="INFO"/>

                              <formatter>

                                  <pattern-formatter pattern="%K{level}%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n"/>

                              </formatter>

                          </console-handler>

                          <periodic-rotating-file-handler name="FILE" autoflush="true">

                              <formatter>

                                  <pattern-formatter pattern="%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n"/>

                              </formatter>

                              <file relative-to="jboss.server.log.dir" path="server.log"/>

                              <suffix value=".yyyy-MM-dd"/>

                              <append value="true"/>

                          </periodic-rotating-file-handler>

                          <logger category="com.arjuna">

                              <level name="WARN"/>

                          </logger>

                          <logger category="org.apache.tomcat.util.modeler">

                              <level name="WARN"/>

                          </logger>

                          <logger category="org.jboss.as.config">

                              <level name="DEBUG"/>

                          </logger>

                          <logger category="sun.rmi">

                              <level name="WARN"/>

                          </logger>

                          <logger category="jacorb">

                              <level name="WARN"/>

                          </logger>

                          <logger category="jacorb.config">

                              <level name="ERROR"/>

                          </logger>

                          <root-logger>

                              <level name="INFO"/>

                              <handlers>

                                  <handler name="CONSOLE"/>

                                  <handler name="FILE"/>

                              </handlers>

                          </root-logger>

                      </subsystem>

                      <subsystem xmlns="urn:jboss:domain:deployment-scanner:1.1">

                          <deployment-scanner path="deployments" relative-to="jboss.server.base.dir" scan-interval="5000"/>

                      </subsystem>

                      <subsystem xmlns="urn:infinispan:server:endpoint:6.1">

                          <hotrod-connector socket-binding="hotrod" cache-container="clustered">

                              <topology-state-transfer lazy-retrieval="false" lock-timeout="1000" replication-timeout="5000"/>

                          </hotrod-connector>

                          <memcached-connector socket-binding="memcached" cache-container="clustered"/>

                          <rest-connector virtual-server="default-host" cache-container="clustered" security-domain="other" auth-method="BASIC"/>

                      </subsystem>

                      <subsystem xmlns="urn:jboss:domain:datasources:1.2">

                          <datasources/>

                      </subsystem>

                      <subsystem xmlns="urn:infinispan:server:core:6.3" default-cache-container="clustered">

                          <cache-container name="clustered" default-cache="default" statistics="true">

                              <transport executor="infinispan-transport" lock-timeout="60000"/>

                              <distributed-cache name="default" mode="SYNC" segments="20" owners="2" remote-timeout="30000" start="EAGER">

                                  <locking isolation="READ_COMMITTED" acquire-timeout="30000" concurrency-level="1000" striping="false"/>

                                  <transaction mode="NONE"/>

                              </distributed-cache>

                              <distributed-cache name="memcachedCache" mode="SYNC" segments="20" owners="2" remote-timeout="30000" start="EAGER">

                                  <locking isolation="READ_COMMITTED" acquire-timeout="30000" concurrency-level="1000" striping="false"/>

                                  <transaction mode="NONE"/>

                              </distributed-cache>

                              <distributed-cache name="namedCache" mode="SYNC" start="EAGER"/>

                          </cache-container>

                          <cache-container name="security"/>

                      </subsystem>

                      <subsystem xmlns="urn:jboss:domain:jca:1.1">

                          <archive-validation enabled="true" fail-on-error="true" fail-on-warn="false"/>

                          <bean-validation enabled="true"/>

                          <default-workmanager>

                              <short-running-threads>

                                  <core-threads count="50"/>

                                  <queue-length count="50"/>

                                  <max-threads count="50"/>

                                  <keepalive-time time="10" unit="seconds"/>

                              </short-running-threads>

                              <long-running-threads>

                                  <core-threads count="50"/>

                                  <queue-length count="50"/>

                                  <max-threads count="50"/>

                                  <keepalive-time time="10" unit="seconds"/>

                              </long-running-threads>

                          </default-workmanager>

                          <cached-connection-manager/>

                      </subsystem>

                      <subsystem xmlns="urn:jboss:domain:jdr:1.0"/>

                      <subsystem xmlns="urn:infinispan:server:jgroups:6.1" default-stack="${jboss.default.jgroups.stack:udp}">

                          <stack name="udp">

                              <transport type="UDP" socket-binding="jgroups-udp"/>

                              <protocol type="PING"/>

                              <protocol type="MERGE3"/>

                              <protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>

                              <protocol type="FD_ALL"/>

                              <protocol type="VERIFY_SUSPECT"/>

                              <protocol type="pbcast.NAKACK2"/>

                              <protocol type="UNICAST3"/>

                              <protocol type="pbcast.STABLE"/>

                              <protocol type="pbcast.GMS"/>

                              <protocol type="UFC"/>

                              <protocol type="MFC"/>

                              <protocol type="FRAG2"/>

                          </stack>

                          <stack name="tcp">

                              <transport type="TCP" socket-binding="jgroups-tcp"/>

                              <protocol type="MPING" socket-binding="jgroups-mping"/>

                              <protocol type="MERGE3"/>

                              <protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>

                              <protocol type="FD_ALL"/>

                              <protocol type="VERIFY_SUSPECT"/>

                              <protocol type="pbcast.NAKACK2">

                                  <property name="use_mcast_xmit">false</property>

                              </protocol>

                              <protocol type="UNICAST3"/>

                              <protocol type="pbcast.STABLE"/>

                              <protocol type="pbcast.GMS"/>

                              <protocol type="MFC"/>

                              <protocol type="FRAG2"/>

                          </stack>           

                      </subsystem>

                      <subsystem xmlns="urn:jboss:domain:jmx:1.3">

                          <expose-resolved-model/>

                          <expose-expression-model/>

                          <remoting-connector/>

                      </subsystem>

                      <subsystem xmlns="urn:jboss:domain:modcluster:1.2">

                          <mod-cluster-config advertise-socket="modcluster" connector="ajp" excluded-contexts="console">

                              <dynamic-load-provider>

                                  <load-metric type="busyness"/>

                              </dynamic-load-provider>

                          </mod-cluster-config>

                      </subsystem>

                      <subsystem xmlns="urn:jboss:domain:naming:1.4"/>

                      <subsystem xmlns="urn:jboss:domain:remoting:1.1">

                          <connector name="remoting-connector" socket-binding="remoting" security-realm="ApplicationRealm"/>

                      </subsystem>

                      <subsystem xmlns="urn:jboss:domain:security:1.2">

                          <security-domains>

                              <security-domain name="other" cache-type="infinispan">

                                  <authentication>

                                      <login-module code="Remoting" flag="optional">

                                          <module-option name="password-stacking" value="useFirstPass"/>

                                      </login-module>

                                      <login-module code="RealmUsersRoles" flag="required">

                                          <module-option name="usersProperties" value="${jboss.server.config.dir}/application-users.properties"/>

                                          <module-option name="rolesProperties" value="${jboss.server.config.dir}/application-roles.properties"/>

                                          <module-option name="realm" value="ApplicationRealm"/>

                                          <module-option name="password-stacking" value="useFirstPass"/>

                                      </login-module>

                                  </authentication>

                              </security-domain>

                              <security-domain name="jboss-web-policy" cache-type="infinispan">

                                  <authorization>

                                      <policy-module code="Delegating" flag="required"/>

                                  </authorization>

                              </security-domain>

                          </security-domains>

                      </subsystem>

                      <subsystem xmlns="urn:jboss:domain:threads:1.1">

                          <thread-factory name="infinispan-factory" group-name="infinispan" priority="5"/>

                          <unbounded-queue-thread-pool name="infinispan-transport">

                              <max-threads count="25"/>

                              <keepalive-time time="0" unit="milliseconds"/>

                              <thread-factory name="infinispan-factory"/>

                          </unbounded-queue-thread-pool>

                      </subsystem>

                      <subsystem xmlns="urn:jboss:domain:transactions:1.4">

                          <core-environment>

                              <process-id>

                                  <uuid/>

                              </process-id>

                          </core-environment>

                          <recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager"/>

                          <coordinator-environment default-timeout="300"/>

                      </subsystem>

                      <subsystem xmlns="urn:jboss:domain:web:1.5" default-virtual-server="default-host" native="false">

                          <connector name="http" protocol="HTTP/1.1" scheme="http" socket-binding="http"/>

                          <connector name="ajp" protocol="AJP/1.3" scheme="http" socket-binding="ajp"/>

                          <virtual-server name="default-host" enable-welcome-root="false">

                              <alias name="localhost"/>

                              <alias name="example.com"/>

                          </virtual-server>

                      </subsystem>

                  </profile>

                     <interfaces>

                  <interface name="site-local">

                     <site-local-address/>

                  </interface>

              </interfaces>

                  <socket-binding-group name="ha-sockets" default-interface="site-local" port-offset="${jboss.socket.binding.port-offset:0}">

                      <socket-binding name="management-native" interface="site-local" port="${jboss.management.native.port:9999}"/>

                      <socket-binding name="management-http" interface="site-local" port="${jboss.management.http.port:9990}"/>

                      <socket-binding name="management-https" interface="site-local" port="${jboss.management.https.port:9443}"/>

                      <socket-binding name="ajp" port="8009"/>

                      <socket-binding name="hotrod" interface="site-local" port="11222"/>

                      <socket-binding name="http" port="8080"/>

                      <socket-binding name="https" port="8443"/>

                      <socket-binding name="jgroups-mping" port="0" multicast-address="${jboss.default.multicast.address:234.99.54.14}" multicast-port="45700"/>

                      <socket-binding name="jgroups-tcp" port="7600"/>

                      <socket-binding name="jgroups-tcp-fd" port="57600"/>

                      <socket-binding name="jgroups-udp" interface="site-local" port="55200" multicast-address="${jboss.default.multicast.address:234.99.54.14}" multicast-port="45688"/>

                      <socket-binding name="jgroups-udp-fd" port="54200"/>

                      <socket-binding name="memcached" interface="site-local" port="11211"/>

                      <socket-binding name="modcluster" port="0" multicast-address="224.0.1.115" multicast-port="23364"/>

                      <socket-binding name="remoting" port="4447"/>

                      <socket-binding name="txn-recovery-environment" port="4712"/>

                      <socket-binding name="txn-status-manager" port="4713"/>

                  </socket-binding-group>

              </server>

              • 4. Re: How i can to connect two or more JDG nodes (cluster mode) in differents machines??
                sebastian.laskawiec

                In that case I'm suspecting that there are some network problems between those hosts.

                 

                If you don't add any options during the startup - you're using UDP configuration (please take a look at this configuration fragment: <subsystem xmlns="urn:infinispan:server:jgroups:6.1" default-stack="${jboss.default.jgroups.stack:udp}">). Please make sure that IP Multicasting is enabled in your network and the firewall has proper configuration to allow UDP traffic on ports 55200, 45688 and 54200. Alternatively, you may switch into TCP mode by changing the default-stack configuration option to tcp. In this case please make sure that the firewall has proper configuration for ports 7600 and 57600.

                 

                Please take a look also at this links:

                • 5. Re: How i can to connect two or more JDG nodes (cluster mode) in differents machines??
                  andresperez0828

                  Hi Sebastian,

                   

                   

                  Thank you so much, the problem was the networks policy for the connection with UDP protocol, once fixed, the nodes between differents machines can communicate.

                   

                   

                  Best Regards,

                   

                   

                  Andrés Pérez

                  • 6. Re: How i can to connect two or more JDG nodes (cluster mode) in differents machines??
                    sebastian.laskawiec

                    No problem! Happy to help!