13 Replies Latest reply on Jul 17, 2011 3:13 PM by into_java

    Issue with HornetQ producer throughput

    nkarthik82

      I am using HornetQ for my application. In HornetQ user manual, it was mentioned that it is better to reuse the same session, producer to improve the performance. So, I wrote a program which will post messages to queue using the same producer.

      The producer performance is good if there are not much concurrent requests. But, When I hit the program with 50 concurrent threads, time taken to post each message in the queue increases becase the same producer is used to send message and i think it gets blocked when 50 concurrent threads post message.

      For ex: when I post 50 messages in 1 second, i.e. 1 message every 20ms, the performance is good. But, when I post 50 messages at the same time, it takes on an average 230ms to post each message in the queue.

      I have attached the complete program (QueuePopulator.java).

      In the program, I am posting messages to the queue in following method:

      public void sendMessagetoQ(String message) {

              TextMessage textMessage;

              try {

                  textMessage = session.createTextMessage();

                  textMessage.setText(message);

                  long startTime = Calendar.getInstance().getTimeInMillis();

                  messageProducer.send(textMessage);

                  System.out.println("time taken - "+ (Calendar.getInstance().getTimeInMillis() - startTime));

              } catch (JMSException ex) {

                  // Log an error and ignore the message.

                  ex.printStackTrace();

              }

          }

       

      I am using HornetQRAConnectionFactory for posting messages to HornetQ.

      JBoss version is 6.0.0.Final and HornetQ version is 2.2.2.Final.

       

      Can someone please tell me whether I am following the right approach of using same producer or is there a better way to improve the performance?

       

      Thanks in advance.

        • 1. Re: Issue with HornetQ producer throughput
          ataylor

          How is the client deployed, the reason i ask is because the connection factory you are using is a jca pooled connection factory, so you arent actually recreating sessions etc when you create them, your probably bound by the number of connections in the pool

          • 2. Re: Issue with HornetQ producer throughput
            nkarthik82

            This client is deployed in a webapp and I am posting messages to client from a servlet.

            So, it is inside a JEE container.

             

            Do you mean, even if i call createConnection, createSession and createProducer for each request that comes to sendMessagetoQ method, it will not actually create a new connection or session or producer?

            I thought only connections are pooled in jca pooled connection factory. I am not sure whether sessions and producers are also pooled in jca pooled connection factory.

            • 3. Re: Issue with HornetQ producer throughput
              ataylor

              connections and sessions are all pooled in JCA,  producers arent but they are quite light weight in comparison. If your pool size is 15 (im guessing here) then that will be what limits your throughput.

               

              Also how are you using concurrent threads in JEE

              • 4. Re: Issue with HornetQ producer throughput
                nkarthik82

                I have given a max-pool-size of 10 for HornetQRAConnectionFactory.

                My question is whether this max-pool-size value covers both connection and session pooling or is there any specific tag to specify session pool size?

                 

                I am using JMeter to post 50 concurrent requests to the servlet which in turn invokes the queue client. I have added timers in the queue client program.

                If I post 50 concurrent requests, I can see that the 50th request takes around 500-600ms to post the message to queue.

                • 5. Re: Issue with HornetQ producer throughput
                  ataylor

                  I have given a max-pool-size of 10 for HornetQRAConnectionFactory.

                  My question is whether this max-pool-size value covers both connection and session pooling or is there any specific tag to specify session pool size?

                  Each connection has one session, remember in JCA ony 1 connection can be used at any one time by any component.

                   

                  I am using JMeter to post 50 concurrent requests to the servlet which in turn invokes the queue client. I have added timers in the queue client program.

                  If I post 50 concurrent requests, I can see that the 50th request takes around 500-600ms to post the message to queue.

                   

                  altho you are invoking 50 concurrent requests onl 10 will be to obtain a connection at any one time, so you arent really testing throuhput.

                  • 6. Re: Issue with HornetQ producer throughput
                    nkarthik82

                    Till now I have been using the same session and producer to test. So, I was not able to achieve the throughput. I will increase the pool size to 50 and test it.

                     

                    And also in code, I will make the following changes:

                    1) create a connection for each request.(comes from pool)

                    2) create a session for each request (comes from pool)

                    3) create a producer for each request(new object will be created)

                    4) post the message to queue.

                    5) close the session

                    6) close the connection.

                     

                    Can you please confirm whether the above steps are correct?

                    And also what do you mean by only one connection can be used by one component at any point of time? If I have 50 concurrent request, each request should be able to use one connection..right?

                    If I set the pool size to 50 and follow the above steps, i think i should be able to achieve a throughput of 50.

                    • 7. Re: Issue with HornetQ producer throughput
                      ataylor

                      Yes, but remember that just be upping your pool size you may not see any increae, there will be other factors like other AS pooling issues for servlets etc.

                       

                      what is it you are actually tying to measure, im assuming you are doing some sort of performance testing

                      • 8. Re: Issue with HornetQ producer throughput
                        nkarthik82

                        yes...I am trying to test the queue performance.

                        I have a pool of 50 listeners and I tried to post 50 concurrent requests to the client. But, the client was too slow.

                        I will try this solution and test the performance.

                        Thank you Andy for your inputs.

                        I will get back to you if I find any issue with this solution.

                        • 9. Re: Issue with HornetQ producer throughput
                          ataylor

                          well you arent actually testing the performance of the queue, you are testing the performance of servlets/JCA/transactions/security and what ever other non JMS related stuff you are using. If you want to testthe performance use standard JMS clients and normal connection factories.

                          • 10. Re: Issue with HornetQ producer throughput
                            nkarthik82

                            Actually, I am trying to test the complete round trip i.e. the time taken for 50 concurrent servlet request to post a message to the queue and return a response to JMeter

                            • 11. Re: Issue with HornetQ producer throughput
                              nkarthik82

                              After making the above changes, time taken to post the message to queue has improved. Queue performance is very good.

                              I am using a singleton QueuePopulator to populate messages from a servlet. When i post 50 concurrent requests to the servlet,  queue performance is fast, but overall time taken is i.e. client->servlet->queue->servlet->client is more.

                               

                              I think with a singleton QueuePopulator, may be the servlet gets blocked for sometime eventhough queue producer is fast. (just guessing)

                               

                              Is it better to use a singleton QueuePopulator class to populate messages in the queue or create more QueuePopulator classes?

                              • 12. Re: Issue with HornetQ producer throughput
                                ataylor

                                I cant really answer that as it depends on too many other factors,network speed, disk, servlet pool etc etc. I would make this configurable so you can tune it when you come to deploy.

                                • 13. Re: Issue with HornetQ producer throughput
                                  into_java

                                  Hi Karthik,

                                  I have run some tests like you in which

                                  Making connection per message is anti-patter.

                                  1. I made session per message (bad idea)

                                  2. Made producer per message (better than above) making session reuse. Avoid anti-pattern as mentioned in the book.

                                   

                                  I then tuned by changing the message to be non durable..Some more improvement

                                  and then finally i think i set the message ack handler for async message ack.