2 Replies Latest reply on Apr 13, 2012 9:33 AM by jatinsut

    Solving JMS message sequence problem in a clustered environment with Aggregator service

    jatinsut

      Hello,

       

       

      I am pretty new to JBoss ESB and just came across few links on it that talked of aggregator (of JMS messages) capabilities. I was wondering id this can be utilized in my application to solve following -

       

       

      I have  a clustered J2EE application, that comprises of eJB's+MDB's, Queues, Spring and Hibernate, that can be deployed to multiple app servers. One use case is huge number of messages having keys getting pumped into a queue and then these messages are picked up my MDB's. Since my application is clustered, any MDB instance on any server of the cluster can pick up the message and process it and so its possible messaaages that have arrived later on queue can be processed earlier since they are picked up by MDB instances on a superior machine and so it proocesses earlier .

       

       

      The objective is the messages having the same keys are picked up and processed in the same sequence in which they are pumped into the Queue. Of course, this doesn't work in a clustered environment and we have resorted to a singleton mDB picking up all messages and processing it and thereby reducing performance.

       

       

      So the question is can I use JBoss ESB aggregation service (reading from my queue) that will aggregate messages on basis of correlation id ( set to key of messages) and then pump the aggregated message (comprising of messages with same key) in to another JMS queue so that MDB instances in a clustered environment reads a bunch of messages(aggregated message by ESB aggregator) in one go and thereby improving performance.

       

       

      Thanks,

      Jatin

        • 1. Re: Solving JMS message sequence problem in a clustered environment with Aggregator service
          tcunning

          Hi Jatin,

           

          The aggregator isn't an aggregator of JMS messages.   It's just an aggregator of messages (irrespective of transport), and it is generally used along with a splitter - where you split a message with a number of records into N number of messages, do some processing on each of N messages, and then aggregate all the record-messages back up when processing is complete.    In 4.11, a new streaming aggregator has been introduced that takes the message sequence number (between 1 and the max message count) and a unique ID.      An aggregation is released when the message count is sent to the aggregator along with the unique ID and all messages in the sequence have been received.

           

          For you to get the aggregator to do what you want it to :

           

          - you'd have to host the service containing it on a single node of your cluster

          - you'd have to have some way of knowing which message in the CORRELATION-ID sequence each message is

          - you'd have to at some point know the total number of messages in the aggregation

           

          Check out the streaming_aggregator quickstart in 4.11 for more details.

           

          --Tom

          • 2. Re: Solving JMS message sequence problem in a clustered environment with Aggregator service
            jatinsut

            Thanks Tom for your prompt reply.

             

            However, if I were to host the service on a single node of my cluster, i would introduce a singe point of failure, not to mention a serialized process of aggegating messages. I am not too sure how much performance gain can be got vis-a-vis the singleton MDB approach that we have right now to read all the messages.

             

            Thanks anyways -

            Jatin