10 Replies Latest reply on Jun 21, 2012 3:11 PM by ethan.c

    How to consume messages from clustered queue?

    ethan.c

      Hi,

       

      I have a clustered environment with 2 nodes(node1 and node2), and both node1 and node2 have a queue named "exampleQueue". And i want to use the UDP connection to access this group.

       

      Like this:

       

               DiscoveryGroupConfiguration groupConfiguration = new DiscoveryGroupConfiguration("231.7.7.7", 9876);  

               ConnectionFactory cf = (ConnectionFactory)HornetQJMSClient.createConnectionFactoryWithoutHA(groupConfiguration, JMSFactoryType.CF);

               Queue queue = HornetQJMSClient.createQueue("exampleQueue");

               .........

       

      The sending message part is working perfectly. If i send 10 message to the queue by using the UDP connection, 5 of them will go to the node1 and the other 5 will go to node2 according to the round robin fashion.

       

      The problem is the consumer: if i create one consumer based on this UDP connection, it will be created on either node1 or node2(i can observe from jconsole). Suppose the consumer is on the node1, it can consume all these messages from node2, but it can't consumer message form node2.  This is not i want, what i want is this cluster of hornetq servers is totally transparent to clients(my client is PHP and listener is JAVA).  It should be like clients send request messages to queue1 and get response messages from queue2, and it does not need to care which node should be going to, just use the common cluster connection.

       

      My real scenario is like this:

      1>. PHP(STOMP) sends a request message to request queue, and waits for the response from the response queue;

      2>. JAVA has listeners to listen this request queue (based on spring listener container).

      3>. After gets this message, JAVA will process this message(bussiness logic, database operations...);

      4>. Then JAVA will send the response message to the response queue;

      5>. And the PHP will get the response message from the response queue(using selector).

       

      So my question is: is that possible for a consumer to consume(select) messages based on cluster level not node level?   I should just care about the clusted connection(Like UDP).

       

      One more question:  Can PHP(STOMP) use UDP connection(231.7.7.7) to access to clusted hornetq sever?  I never try this, i wanna confirm.

       

      Thanks in advance!

        • 1. Re: How to consume messages from clustered queue?
          ataylor

          The sending message part is working perfectly. If i send 10 message to the queue by using the UDP connection, 5 of them will go to the node1 and the other 5 will go to node2 according to the round robin fashion.

          that makes sense to me, messages will be round robined around the cluster.

           

           

          The problem is the consumer: if i create one consumer based on this UDP connection, it will be created on either node1 or node2(i can observe from jconsole).

          again, that makes sense, it will consume messages depending on the state of the cluster at the time it is conected.

           

          Im not sure what you are asking here, remember that messages are distributed at the etime of sending not at the time the consumer attaches, maybe you should look at some of the redistribution configuration

          • 2. Re: How to consume messages from clustered queue?
            ethan.c

            Thanks for your relpy, Andy!

             

            Let me clarify my scenario:

            1> I have 2 queues on each node:  "queue.request"  and "queue.response". Sent to request queue and get result from response queue.

            2> Client(STOMP) sends a request message to queue "queue.request" and then goes to "queue.response" to try to get response message.

            3> Of course, there is a listener(JAVA) in backend to listen request queue and send result to the response queue.

             

                                send and get message by selecter

            php(client)<------------------------------------------------------->-hornetq server cluster<------------------------------->java listener and producer

             

            The problem happens when Clients create a consumer and try to select message from response queue by correlationid. I don't know which node the response message will be in, and that makes sense because it depends on the clustering strategy.  So if the cusumer i created is not on the queue which my response message is existing in, the cusume couldn't get the message.

            I do understand "messages are distributed at the etime of sending not at the time the consumer attaches" , but this is not i want.  I want my consumer can consume messages from all the nodes transparently, because it's connecting to the group address(UDP).  Is that possible for Hornetq?    

            Or am i using queue in wrong way?  I thought it's pretty general scenario.

             

            Thanks!

            .

            • 3. Re: How to consume messages from clustered queue?
              ethan.c

              If there are total 10 messages on the response queue, 5 in node1 and 5 in node2.   PHP client creates a consumer(not a listener) with selector by corralationid and  try to get the correct message.  Obviously i want the consumer to select message from the 10 messages rather than 5.

              • 4. Re: How to consume messages from clustered queue?
                clebert.suconic

                There's a JIRA to use filters to redistribute message. Right now the only thing that makes it to redistribute is the existence of a consumer or not.

                 

                I would probably recommend using a durable queue with the filter you want instead of a filter on a whole queue for this kind of thing.

                • 5. Re: How to consume messages from clustered queue?
                  ethan.c

                  I solved this problem.

                  There are some coniguration wrong in my previous setting.  I reset the "forward-when-no-consumers" to false and "redistribution-delay" to 0, and it's working as i want. Now it behaves in this way:

                  There are 2 nodes in my cluster: node1 and node2, and each of them has a queue named "queue.response" .  Suppose there are 5 message in node1 and 5 messages in node2.  Then i create a consumer trying to consume a messagge, it will attach to one of node(either node1 or node2).  If it attaches to node1, then 1 message gets consumed from node1 and these 5 messages in node2 also get forwarded to node1. So there are 9 messages left in node1 and 0 messages left in node2.   This is the way i want, as consumers can alway get the messgaes.

                   

                  Thanks!

                  • 6. Re: How to consume messages from clustered queue?
                    into_java

                    Hi Ethan

                    Can you share a code snippet of your consumer and how do you bind your consumer to consume for queue from node 1 as well as from node 2.

                     

                    Thanks

                    • 7. Re: How to consume messages from clustered queue?
                      jbertram

                      I believe that Ethan is saying that the server is redistributing messages in the background so that a single consumer may connect to one node and consume all the messages from both nodes.

                      • 8. Re: How to consume messages from clustered queue?
                        into_java

                        Hi Jusin, ethan

                        Well i would really like to know what is that configuartion which makes the consumer on node 1 not just consume the messages from queue1 on node1 as well as queue1 on node2.

                         

                        I have a same scenario where 10 messages are spread across 2 nodes each having 5 messages.

                         

                        But the consumer i have on node 1 is only able to get 5 messages while the consumer on node 2 is only getting 5 messages.

                         

                        the queue is configured to be clustered.

                         

                        Can you give me some pointers.

                         

                        Thanks

                        • 9. Re: How to consume messages from clustered queue?
                          ataylor

                          But the consumer i have on node 1 is only able to get 5 messages while the consumer on node 2 is only getting 5 messages.

                          That sounds correct, they are evenly distributed because you have 2 consumers, if you only had 1 consumer it would receive all 10.

                           

                          If you are struggling with understanding clustering then read the user manual and look at one of the many clustering examples we have.

                          • 10. Re: How to consume messages from clustered queue?
                            ethan.c

                            Justin is correct that the server is redistributing messages in the background.

                             

                            This is my configuration:

                             

                            <cluster-connections>

                                  <cluster-connection name="mq-cluster1">

                                     <address>jms</address>    

                                     <connector-ref>netty</connector-ref>

                                     <retry-interval>500</retry-interval>

                                     <use-duplicate-detection>true</use-duplicate-detection>

                                     <forward-when-no-consumers>false</forward-when-no-consumers>

                                     <max-hops>1</max-hops>

                                     <discovery-group-ref discovery-group-name="dg-group1"/>

                                  </cluster-connection>

                            </cluster-connections>

                            ....

                            <address-setting match="#">

                                     <dead-letter-address>jms.queue.DLQ</dead-letter-address>

                                     <expiry-address>jms.queue.ExpiryQueue</expiry-address>

                                     <redelivery-delay>0</redelivery-delay>     

                                     <message-counter-history-day-limit>10</message-counter-history-day-limit>

                                     <max-size-bytes>104857600</max-size-bytes>

                                     <address-full-policy>PAGE</address-full-policy>

                                     <page-size-bytes>10485760</page-size-bytes>

                                     <redistribution-delay>0</redistribution-delay>

                            </address-setting>