3 Replies Latest reply on Nov 16, 2010 10:23 AM by timfox

    Deep queues and publish throughput

    jsboulanger
    jsboulanger Nov 16, 2010 4:30 AM

      Hi,

       

      I'm new to HornetQ and I'm evaluating it for a project. One of our scenarios is to create a deep queue by publishing millions of messages without any processes  consuming them. When we start our publishers we get a publish rate of about 17,000 messages per second, but within a few minutes it decreases to about 200 messages per second.

      Then when I start some consumers there consumption rate is increadibly low (a few messages per second) that rate increases as the depth of the queue decreases, to eventually reach thousands of messages per sec.

       

      I didn't investigate deeper yet, but I imagine this has to do with the journaling or paging strategy?

       

      I was wondering if anyone could help me understand what is going on or point me to the right information?? I was also wondering if there is anyway on improving that with some configuration or tuning... maybe I'm missing something?

       

      I used 2.1.2.Final on CentOS with AIO enabled. The system contains a single queue. Publishers and consumers all speak STOMP. I get the same kind of behavior with persistent and non-persistent messages.

       

      Thank you in advance,

      Jean-Sebastien

        • 1. Re: Deep queues and publish throughput
          timfox
          timfox Nov 16, 2010 8:18 AM (in response to jsboulanger)

          Most discs can do about 200 syncs per sec. That would be a hardware limit, not specific to HornetQ.

           

          Disc synching and persistent performance has been discussed numerous times on other threads. It's worth a trawl.

            • Actions
          • 2. Re: Deep queues and publish throughput
            jsboulanger
            jsboulanger Nov 16, 2010 8:50 AM (in response to timfox)

            I got those results for NON-persistent messages. Does that mean HornetQ syncs to disc for every single message (even non-persistent ones) once it starts paging?

             

            I also ran ActiveMQ on the same hardware and I don't get the same behavior at all, although the initial publish rate is less than HornetQ, it does stays constant to about 1,200 messages per second, even when I have several gigabytes of messages stored in the queue.

             

            HornetQ starts with a very high throughbut 17,000 msg/s and progressively (over several minutes) goes down to 200msg/s (and even less).

             

            We also tested other Messaging systems (non-Java) and HornetQ is really the only one that had this degrading behavior with deepening queues. I had the impression it had to do with the way HornetQ handles large queues but I might be wrong.

              • Actions
            • 3. Re: Deep queues and publish throughput
              timfox
              timfox Nov 16, 2010 10:23 AM (in response to jsboulanger)

              It shouldn't be synching for np messages.

               

              It's hard to know what's going on without seeing some kind of test program and config.

                • Actions