12 Replies Latest reply on May 17, 2012 7:14 AM by ataylor

    Colocated Live and Backup servers in HornetQ

    qtm

      Hi,

       

      In the 2.2.2 documentation there's a chapter about application server cluster configurations : http://docs.jboss.org/hornetq/2.2.2.Final/user-manual/en/html/clusters.html . I can't find anything similar in the documentation for 2.2.5. Am I missing something or was this part removed because of some issues?

       

      I would like to use it with the latest Jboss 7, so please tell me if it's safe to do so.

       

      Regards

        • 1. Re: Colocated Live and Backup servers in HornetQ
          ataylor

          yeah, that looks to have been taken out, not sure why, but yeah, your safe to do so. I would use 2.2.14 or AS7 if you can

          • 2. Re: Colocated Live and Backup servers in HornetQ
            qtm

            Hi Andy,

             

            Thanks for answering. I want to double-check if I understand correctly the way this configuration works:

             

            In the simple context of a JBoss(J1) instance with HQLiveserver1(HQL1) and HQbackupserver(HQB2 - for HQL2) and of an other JBoss(J2) instance with HQLiveserver2(HQL2) and HQbackupserver(HQB1 - for HQL1).

             

            - if I have "clustered" queues/topics I shouldn't redefine them. If they exist in the configuration for HQL1 they shouldn't be in the configuration for HQB2, because of JNDI name clashes. Let's assume HQL2 goes down and HQB2 becomes live. I can access the queue by JNDI or by TransportConfiguration&HornetQJMSClient. By JNDI I'm getting the queue defined for HQL1 and almost everything is ok. Almost everything - because I can't browse HQB2. If I use the TC&HQJMSClient I can do all I want.

            - everything that's not clustered should be defined in the back-up too.

            - I'm using just one port for remoting per Jboss instance. To me it seems the right way to it. Should I define another port for remoting for the back-up?  For messaging I have defined them :


            <socket-binding name="messaging" port="5445"/>

            <socket-binding name="messaging-backup" port="5545"/>

            <socket-binding name="messaging-throughput" port="5455"/>

            <socket-binding name="messaging-throughput-backup" port="5555"/>

            - I'm using static discovery. Is it better to use dynamic discovery?

            - Except <jmx-domain>org.hornetq.backup1</jmx-domain> is there any other isolation that I should take notice of?

             

             

            Thanks again

            • 3. Re: Colocated Live and Backup servers in HornetQ
              qtm

              Hello,

               

              I'm still puzzled about the JNDI use with colocated servers. Should I completely disregard JNDI when working with collocated servers and work directly with the messaging via the messaging port?

               

              For the next use case I have no solution : A cluster with 2 nodes - each of them having a back-up for the other. One node goes down, so on one jboss instance I'll have 2 hornetq live servers running.

              If I want to use JNDI to browse a queue, I'll only get the messages on one hornetq servers. Also, if a have a queue specific to the second node(the one that goes down and for which I have the back-up defined on the running node), I can't access it through JNDI if I don't expose it in the back-up configuration(and I've read that the messaging components shouldn't be because they're created from the journal).

              • 4. Re: Colocated Live and Backup servers in HornetQ
                gaohoward

                If you don't want JNDI you can directly create the connection factories. I think it is said in the user-manual. Check it out yourself.

                 

                Howard

                • 5. Re: Colocated Live and Backup servers in HornetQ
                  qtm

                  Hi,

                   

                  No, I want JNDI, the problem is that I don't see it covering all the cases. I've described such a case in the previous post

                  • 6. Re: Colocated Live and Backup servers in HornetQ
                    ataylor

                    You can use JNDI:

                     

                    Any clients connected to the live server that is going to crash will failover to the backup server on the other node. This node then clusters with the live server on its own node and redsitributes any messages to the live server if it needs to (if your using MDB's etc). New clients should always connect to the live server ont the backup that has become live so there is no issue with JNDI here.

                    • 7. Re: Colocated Live and Backup servers in HornetQ
                      qtm

                      Ok,

                      Node 1 - has a queue - testQueue, Node 2 - has 2 queues - testQueue, testQueue2. So the configuration on node 1 (includes the back-up for node2) should look like:

                       

                      <subsystem xmlns="urn:jboss:domain:messaging:1.1">

                                  <hornetq-server>

                                        ....              

                                         <jms-connection-factories>

                                         ...

                                          <connection-factory name="RemoteConnectionFactory">

                                              <connectors>

                                                  <connector-ref connector-name="netty"/>

                                              </connectors>

                                              <entries>

                                                  <entry name="RemoteConnectionFactory"/>

                                                  <entry name="java:jboss/exported/jms/RemoteConnectionFactory"/>

                                              </entries>

                                              <ha>true</ha>

                                              <reconnect-attempts>3</reconnect-attempts>

                                          </connection-factory>

                                        ...                 

                                      </jms-connection-factories>

                                      <jms-destinations>

                                          <jms-queue name="testQueue">

                                              <entry name="queue/test"/>

                                              <entry name="java:jboss/exported/jms/queue/test"/>

                                          </jms-queue>

                                      </jms-destinations>

                      ...                

                              </hornetq-server>

                                  <hornetq-server name="backup_for_server_2">

                                     ...

                                     <backup>true</backup>

                                    ...

                                //NO JMS_CONNECTION FACTORIES         

                                      <jms-destinations>

                                          <jms-queue name="testQueue2">

                                              <entry name="queue/test2"/>

                                              <entry name="java:jboss/exported/jms/queue/test2"/>

                                          </jms-queue>

                                      </jms-destinations>

                           ...

                              </hornetq-server>

                      • 8. Re: Colocated Live and Backup servers in HornetQ
                        ataylor

                        I'm not sure what your point is with your last post, could you be more specific.

                        • 9. Re: Colocated Live and Backup servers in HornetQ
                          qtm

                          Hi,

                           

                          I'm trying to use JNDI to browse my queues(clustered or not) in a colocated environment. I've attached my configurations. I'm having 2 jboss(7.1.1) instances on my local computer, each of the jboss instances having a hornetq live server and a backup.

                           

                          Jboss instance 1:

                          - host : [my_ip]

                          - remoting port : 4447

                          - messaging port : 5445

                          - messaging back-up port : 5545

                          - hornetq live server 1

                          - hornetq backup server 2

                          - queues : testQueue, testQueue2

                           

                          Jboss instance 2:

                          - host: [my_ip]

                          - remoting port : 4448

                          - messaging port : 5446

                          - messaging back-up port : 5546

                          - hornetq live server 2

                          - hornetq backup server 1

                          - queues : testQueue, testQueue2, testQueue3

                           

                          hornetq live server 1 and hornetq backup server 1 share the same file system. So do hornetq backup server 2 and hornetq live server 2

                          testQueue and testQueue2 are clustered, testQueue3 is not.

                           

                          If I produce some messages on testQueue when both nodes are running, the messages will be load -balanced as expected. So if I browse testQueue on Jboss instance 1 I see half of the messages and if I browse testQueue on Jboss instance 2 I see the other half. Using JNDI I connect to Jboss instance 1 on 4447 and to Jboss instance 2 on 4448. When one of nodes goes down,

                          the back-up becomes live, but I will have only one remoting port to connect to (4447 or 4448). Browsing using JNDI will give me just the messages on the live server. I can't see the other half of the messages, because the back-up server that has become live has no individual remoting port.

                          If I browse directly without JNDI I can connect to the messaging back-up ports and I can see all the messages. My problem is that I don't know how to define a remoting port just for the back-up.

                           

                          I browse using JNDI like this:

                           

                                   final Properties env = new Properties();

                                      env.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");       

                                      env.put(Context.PROVIDER_URL, "remote://[my_ip]:4447");

                                      env.put(Context.SECURITY_PRINCIPAL, "admin");

                                      env.put(Context.SECURITY_CREDENTIALS, "root");

                           

                                      Context ctx = new InitialContext(env);

                                      Queue q = (Queue)ctx.lookup("jms/queue/test");       

                                      ConnectionFactory factory = (ConnectionFactory)ctx.lookup("/jms/RemoteConnectionFactory");

                           

                                      QueueConnection con = null;

                                      try{

                                          con = (QueueConnection)factory.createConnection("admin", "root");

                                          con.start();

                           

                                          TestUtils.showMessages(con, q);

                                      }... finally{....}

                           

                           

                           

                          I browse without JNDI like this:

                           

                          Map<String, Object> map11 = new HashMap<String, Object>();

                                  map11.put("host", "[my_ip]");

                                  map11.put("port", 5445);

                           

                                  Map<String, Object> map12 = new HashMap<String, Object>();

                                  map12.put("host", "[my_ip]");

                                  map12.put("port", 5546);

                           

                                  Map<String, Object> map21 = new HashMap<String, Object>();

                                  map21.put("host", "[my_ip]");

                                  map21.put("port", 5446);

                           

                                  Map<String, Object> map22 = new HashMap<String, Object>();

                                  map22.put("host", "[my_ip]");

                                  map22.put("port", 5545);

                           

                                  TransportConfiguration config11 = new TransportConfiguration(NettyConnectorFactory.class.getName(), map11);

                                  TransportConfiguration config12 = new TransportConfiguration(NettyConnectorFactory.class.getName(), map12);

                                  TransportConfiguration config21 = new TransportConfiguration(NettyConnectorFactory.class.getName(), map21);

                                  TransportConfiguration config22 = new TransportConfiguration(NettyConnectorFactory.class.getName(), map22);

                           

                                  ConnectionFactory cf1 = null;

                                  QueueConnection con1 = null;

                                  ConnectionFactory cf2 = null;

                                  QueueConnection con2 = null;

                                  try{

                                      cf1 = (ConnectionFactory)HornetQJMSClient.createConnectionFactoryWithHA(JMSFactoryType.QUEUE_CF,

                                              new TransportConfiguration[]{config11, config12});

                                      cf2 = (ConnectionFactory)HornetQJMSClient.createConnectionFactoryWithHA(JMSFactoryType.QUEUE_CF,

                                              new TransportConfiguration[]{config21, config22});

                                      con1 = (QueueConnection)cf1.createConnection("admin", "root");       

                                      con1.start();

                                      con2 = (QueueConnection)cf2.createConnection("admin", "root");       

                                      con2.start();

                           

                                      Queue q1 = HornetQJMSClient.createQueue("testQueue");

                                      Queue q2 = HornetQJMSClient.createQueue("testQueue");

                           

                                      System.out.println("NODE1::::::::::");

                                      TestUtils.showMessages(con1, q1);

                                      System.out.println("NODE2::::::::::");

                                      TestUtils.showMessages(con2, q2);

                           

                           

                          TestUtils:

                          public static void showMessages(Queue q, QueueSession queueSession) throws Exception{

                           

                                  QueueBrowser qb = queueSession.createBrowser(q);

                           

                                  Enumeration<?> en = qb.getEnumeration();

                                  while (en.hasMoreElements()){

                                      Object next = en.nextElement();

                                      if (next instanceof TextMessage){

                                          TextMessage m = (TextMessage) next;

                                          System.out.println(m.getText() + ";Id=" + m.getJMSMessageID());

                                      }

                                      else{

                                          System.out.println(((Message) next).getJMSMessageID());

                                      }

                                  }

                                  qb.close();   

                              }

                           

                              public static void showMessages(QueueConnection queueConnection, Queue queue) throws Exception{       

                           

                                  QueueSession qSession = queueConnection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);

                                  showMessages(queue, qSession);

                                  qSession.close();       

                              }

                          • 10. Re: Colocated Live and Backup servers in HornetQ
                            ataylor

                            if you want to lookup the queues on the backup server then just add a connection factory to the backup config, just make sure it has a different name as jndi is shared across the whole server. A couple of points tho:

                             

                            a) like i said before in a colocated environment new clients shouldn't be connecting to the backup (now live) server, it is there just to deal with clients that have failed over and to (re)distribute messages to the live (real) server.

                            b) using queue browsers in a cluster doesnt make sense as you are browsing only 1 queue in the cluster and since messages are being redistributed all the time you can never get a true picture of what messages are where.

                            c) altho having non clustered queues in a cluster is fine they aren't really supposed to survive crashes (altho the backup server will recreate it and load in its messages), the semantics for this are undefined. If you want a queue available in HA it should be clustered.

                            • 11. Re: Colocated Live and Backup servers in HornetQ
                              qtm

                              Hi,

                              Thank you for your answers and patience.

                               

                              Regarding a) - does it apply to dedicated back-up servers too or is it recommended just for colocated back-ups?

                              • 12. Re: Colocated Live and Backup servers in HornetQ
                                ataylor

                                no its fine in a dedicated backup