12 Replies Latest reply on Aug 29, 2012 2:03 PM by devinderpal

    Does jBPM support clustering for load balancing?

    r-morioka

      Hi all,

       

      Does jBPM  take into consideration about clustering for load balancing?

      In our application,we are thinking about such a model: two jBPM servers, a DB server, a load balaner for load balancing.

      (Please refer to "jBPMClusteringModel.png". )

       

      In this case, would any problem occur?

       

      I confirmed and researched  about the topic of cluster  in jBPM 5.1 forum. But almost all discussions are not closed.

      So if you have any suggestion, please tell me.

       

      Thanks and regards

      Ryoichi Morioka

        • 1. Re: Does jBPM support clustering for load balancing?
          swiderski.maciej

          I would say that it depends

           

          First of all it depends on how are you going to maintain your sessions? Another question is will you have your cluster in "share nothing" strategy or are you considering any caching mechanism?

           

          Could you please elaborate a bit about your case so we would be able to help here a bit, hopefully.

           

          Cheers

          • 2. Re: Does jBPM support clustering for load balancing?
            r-morioka

            Hello Maciej.

            Thank you for your reply.

             

            I explain the detail of app's architecture which I expect.

             

            1)How are you going to maintain your sessions?

            Please refer  to "ClusteringImage1", it is the arquitecher which I want to make.(I added the attached file.)

            The TaskClient and the TaskService is working on each servers and they are using same DB(postgreSQL).

             

            TaskServerSessions are working in each servers and StatefulKnowledgeSessions are made in each servers.

            When web app recieve user's request, web app send the msg to BPM engine. And BPM engine execute "making new session of the flow" or "starting and completing the task".

             

            The load balancer is controlling the request from the user's to webapp.

             

            2)is will you have your cluster in "share nothing" strategy or are you considering any caching mechanism?

            About information of which are stored in jBPM's repositry,  I want to share these information between each servers0. And I think this issue can implement cause same DB is used by each servers.

             

            Other than these I don't want to share in each servers. (If web apps need shareing the information, I will do it. But I don't want to share jBPM informations between each servers.(Other than information which are stored in jBPM's repositry.)

             

            Thanks and regards

            Ryoichi Morioka

            • 3. Re: Does jBPM support clustering for load balancing?
              swiderski.maciej

              Thanks, now I pretty much understood what you try to achieve.

               

              Just one last question - are you going to use persistence of your sessions (StatefulKnowledgeSession)?

              If so, you need to make sure that particular session will be loaded from db only by one server to avoid troubles when persisting sessions (potential overriding each other). You mentioned that:

               

              Morioka Ryoichi wrote:

               

              When web app recieve user's request, web app send the msg to BPM engine. And BPM engine execute "making new session of the flow" or "starting and completing the task".

              so every request will create new session? One option would be to correlate userId with session so every user will have its own session that will be bound to session life cycle of the web application, and problem with same session loaded on both servers will be solved if your load balancer will use "sticky sessions".

               

              Another thing, if you consider to have task service and jbpm engine running in the same JVM, you should take a look at latest implementation of local task service so you will not execute it over the network (for instance mina). It should be released with 5.2 version but it is already available as nightly build.

               

              Hope this helps

              • 4. Re: Does jBPM support clustering for load balancing?
                r-morioka

                Hello Maciej.

                Thank you for your helpful reply.

                 

                I have somequestions for confirming your remark.

                Please teach me if my understanding is wrong.

                 

                1)Do you want to say "We can do jBPM clustering. And when we think about clustering, we have to care about StatefulKnolegeSession."?

                About your last question, I will use StatefulKnolegeSession. And I will make this session instance when making the new flow.

                 

                The system which I want to make is human workflow system. A user submit a petition and other users check the petition.

                So when user grant the petition, I expect changing the task status by using taskClient.

                 

                Maciej Swiderski による書き込み:

                 

                If so, you need to make sure that particular session will be loaded from db only by one server to avoid troubles when persisting sessions (potential overriding each other). You mentioned that:

                 

                From these your discription, I understood If I can controll the session and I can particular one session is loaded only in one server at that time, I can implement cluster model of jBPM. Is my understanding right?

                 

                2)Other than getting informations from DB(StatefulKnoledgeSession), I don't have to share informations between each server, right?

                From your discription, I think I don't have to.  But, I want to confirm.

                 

                Thanks and regards

                Ryoichi Morioka

                • 5. Re: Does jBPM support clustering for load balancing?
                  swiderski.maciej

                  As I would see the configuration:

                  • knowledge base is loaded on both servers from a shared repository, Guvnor for instance using knowledge agents
                  • sessions are maintained per server with persistence, can be reused across requests for the same user (just an example of session reuse)
                  • task service is available for every server as independed service
                  • load balancer is distributing load to app servers based on session affinity so same user will be directed to the server where his/her session is still active

                   

                  Just to summarize, what was discribed on these few posts I think should allow you to cluster jBPM where only db and knowledge base repository are shared between application servers.

                   

                  Good luck

                  • 6. Re: Does jBPM support clustering for load balancing?
                    r-morioka

                    Hello Maciej.

                    Thank you for your helpful reply.

                     

                    Now I can understand how care about the session when I want to use jBPM in cluster.

                    I will try working jBPM in cluster.

                     

                    Thanks and regards

                    Ryoichi Morioka

                    • 7. Re: Does jBPM support clustering for load balancing?
                      swiderski.maciej

                      Glad to hear you find this useful. Please post back to the forum what you have achieved, that could be very valuable input for others

                       

                      Cheers

                      Maciej

                      • 8. Re: Does jBPM support clustering for load balancing?
                        mbuzdugan

                        Hello Maciej

                         

                        The TaskService schedules deadline tasks (code below) and it does that in the constructor.

                         

                        .....................................................................................................................................

                                for (Object object : em.createNamedQuery("UnescalatedDeadlines").getResultList()) {

                                    DeadlineSummary summary = (DeadlineSummary) object;

                                    schedule(new ScheduledTaskDeadline(summary.getTaskId(),

                                            summary.getDeadlineId(),

                                            this),

                                            summary.getDate().getTime() - now);

                                }

                        .....................................................................................................................................

                         

                        If the TaskService is created in each node (server) in the cluster, duplicated deadlines tasks will be created.

                        How can this be avoided?

                         

                        Thanks

                        Regards

                        Marian B

                        • 9. Re: Does jBPM support clustering for load balancing?
                          swiderski.maciej

                          Good point Marian, I admit that I mainly focused on jBPM runtime rather than task service. Please correct me if I am wrong, but wouldn't that be an issue only when requirement is to have task expiration?

                           

                          Anyway, I think this should be done by introducing external scheduler (for instance quartz) that can be clustered so the problem would be partially avoided, of course task service code should be enhanced here as well. Currently I am experimenting with quartz implementation of timer service for jBPM runtime so I will try to take a look at task service too.

                           

                          But you are probably more interested in immediate solution, right? If so, a kind of "quick and dirty" workaround would be to modify named query (UnescalatedDeadlines inside orm file) on one of the cluster members to make sure it will not return any tasks so only one cluster member will be hosting scheduled jobs.

                           

                          HTH

                          Maciej

                          • 10. Re: Does jBPM support clustering for load balancing?
                            mbuzdugan

                            Indeed Maciej, that's exactly what we are doing:

                              -return empty results for "UnescalatedDeadlines" and 

                              -create (clustered) Quartz tasks

                             

                            It is probably better to avoid DB interaction in the ctor anyway. It is convenient to be piggybacked on the ctor, but it isn't critical to the creation of the service and arguably neither relevant.

                            Just an opinion.

                             

                            Thanks again!

                             

                            Regards

                            Marian B.

                            • 11. Re: Does jBPM support clustering for load balancing?
                              mbuzdugan

                              Maciej

                               

                              Wanted to add another thing that you might want to address and also let other know. Perhaps it will also help you with the Quartz work that you are experimenting.

                               

                              If you are scheduling the task with Quartz for example using a global scheduler, that means is persisted, the scheduling would be done in a different transaction.

                              This is because in TaskServiceSession::addTask() the persistence of task data is done in one transaction - doOperationInTransaction(), and the scheduling is done afterwards. So, if the scheduling fails from some reason the task won't be scheduled anymore as it isn't possible to rollback the persistence of task data.

                               

                              There are a couple of alternatives in short term and conceivably in long term (which is what I consider).

                              In long term perhaps this can be addressed inside of the TaskServiceSession (for example by allowing custom persistence activites be included as needed - with diligence of course).

                               

                              Regards

                              Marian B

                              • 12. Re: Does jBPM support clustering for load balancing?
                                devinderpal

                                Instead of implementing this complicated approach for jbpm session management. Can't you just use single StatefulKnowledgeSession across the cluster and replicate it's JVM state by using distributed cache such as infinispan. Only reason you can't use single session across nodes is that it's state gets out of sync, but infinispan can take care of that. I know StatefulKnowledgeSession is not serializable, but I think in infinispan you can plugin a class that can handle marshalling of jbpm session