10 Replies Latest reply on Aug 31, 2010 3:56 AM by aessiari

    Is the simple fork/join with async execution paths broken in jBPM 4.0

      hi All,

       

      We are exploring the simple fork/join construct for an usecase where in the execution path of each of the fork's branch has to be asynchronous. I have provided all the necessary details below, please do let me know if i am missing some thing very fundamental. This seems like a very fundamental usecase and may be we are missing just some very fundamental, please do let us know.

       

      Issue faced:

      The fork node starts the execution of both the branches in parallel by using inbuilt jBPM messaging. We have observed that the job executor picks up both the jobs and starts executing them concurrently. However the workflow never enters the join state.

       

      Environment:

      OS: Linux/Windows

      Java: 1.5.0_22

      jBPM: latest stable 4.0 release from sourceforge.

      Executed as a simple JUnit testcase.

       

      Problem definition:

      We have a bunch of very long running processes that we want to execute in parallel. Once all these parallel processes are done we would like to continue with the workflow. We are trying to model this using fork/join and async continuation.

       

      We are trying to execute the test cases provided in jBPM 4.0 with some minor modifications. For the prototype we are simulating long running processes as simple java nodes. The workflow definition is as follows:

       

      process.jpdl.xml:

      <?xml version="1.0" encoding="UTF-8"?>

       

      <process name="AsyncFork" xmlns="http://jbpm.org/4.0/jpdl">

       

          <start g="22,69,80,40">
              <transition to="fork1" />
          </start>

       

          <fork g="99,68,80,40" name="fork1">
              <transition g="122,41:" to="ship goods" />
              <transition g="123,142:" to="send bill" />
          </fork>

       

          <java class="org.jbpm.examples.async.fork.Application" g="159,17,98,50"
              method="shipGoods" name="ship goods" continue="async">
              <transition g="297,42:" to="join" />
          </java>

       

          <java class="org.jbpm.examples.async.fork.Application" g="159,117,98,50"
              method="sendBill" name="send bill" continue="async">
              <transition g="297,141:" to="join" />
          </java>

       

          <join g="274,66,80,40" name="join">
              <transition to="print join" />
          </join>

       

          <java g="159,117,98,50"
              method="printjoin" name="print join">
              <transition g="297,141:" to="end" />
          </java>
         
          <end g="353,67,48,48" name="end" />

       

      </process>

       

      Application.java: (Note: We have just modified the given java file)

      package org.jbpm.examples.async.fork;

       

      import java.io.Serializable;

       

      /**
      * @author Tom Baeyens
      */
      public class Application implements Serializable {

       

          private static final long serialVersionUID = 1L;

       

          public void shipGoods() {
              // assume automatic calculations here
              for (int i = 10; i > 0; i--) {
                  System.out.println("Shipping goods");
                  sleep();
              }
          }

       

          public void printjoin() {
              System.out.println("Joining both the forked execution paths");
          }
         
          private void sleep() {
              try {
                  Thread.sleep(1000);
              } catch (InterruptedException e) {
                  e.printStackTrace();
              }
          }

       

          public void sendBill() {
              // assume automatic calculations here
              for (int i = 10; i > 0; i--) {
                  System.out.println("New:Sending bill");
                  sleep();
              }

       

          }
      }

       

      AsyncForkTest.java:(Note: We have just modified the given java file)

      package org.jbpm.examples.async.fork;

       

      import java.util.Date;
      import java.util.List;

       

      import org.jbpm.api.Execution;
      import org.jbpm.api.ProcessInstance;
      import org.jbpm.api.job.Job;
      import org.jbpm.test.JbpmTestCase;

       


      /**
      * @author Tom Baeyens
      */
      public class AsyncForkTest extends JbpmTestCase {

       

        String deploymentId;
       
        protected void setUp() throws Exception {
          super.setUp();
         
          deploymentId = repositoryService.createDeployment()
              .addResourceFromClasspath("org/jbpm/examples/async/fork/process.jpdl.xml")
              .deploy();
        }

       

        protected void tearDown() throws Exception {
      //    repositoryService.deleteDeploymentCascade(deploymentId);
         
      //    super.tearDown();
        }

       

        public void testAsyncFork() {
          ProcessInstance processInstance = executionService.startProcessInstanceByKey("AsyncFork");
          String processInstanceId = processInstance.getId();
         
          System.out.println("processInstanceId: " + processInstanceId);
         
          while (true) {       
          }
         
      //    List<Job> jobs = managementService.createJobQuery()
      //      .processInstanceId(processInstanceId)
      //      .list();
      //
      //    assertEquals(2, jobs.size());
      //   
      //    Job job = jobs.get(0);
      //
      //    managementService.executeJob(job.getId());
      //   
      //    job = jobs.get(1);
      //
      //    managementService.executeJob(job.getId());
      //
      //    Date endTime = historyService
      //      .createHistoryProcessInstanceQuery()
      //      .processInstanceId(processInstance.getId())
      //      .uniqueResult()
      //      .getEndTime();
      //
      //    assertNotNull(endTime);
        }
      }

       

      jbpm.cfg.xml: We have enabled jobexecutor.

      <?xml version="1.0" encoding="UTF-8"?>

       

      <jbpm-configuration>

       

        <import resource="jbpm.default.cfg.xml" />
        <import resource="jbpm.tx.hibernate.cfg.xml" />
        <import resource="jbpm.jpdl.cfg.xml" />
        <import resource="jbpm.identity.cfg.xml" />

       

        <!-- Job executor is excluded for running the example test cases. -->
        <!-- To enable timers and messages in production use, this should be included. -->
        <import resource="jbpm.jobexecutor.cfg.xml" />

       

        <import resource="jbpm.mail.templates.examples.xml" />

       

      </jbpm-configuration>

       

      Our Observations and Analysis:

      • 2 jobs gets created in the JBPM4_JOB table one each for the async execution path
      • Job executor picks up both these jobs concurrently and starts executing the java action
      • With debugger on what i have noticed is that each of the job executor threads loads up the ExecutionImpl object into the hibernate session at the start of the execution and rarely hits the DB back for obvious performance reasons.
      • ExecutionImpl object is loaded at the start of thread execution, any of the state changes made by other job executor thread is not visible in the current thread unless the hibernate object is reloaded and this never happens as its a separate hibernate session and thread of execution.
      • Each of the execution paths tries to execute the JoinActivity
      • In the JoinActivity there is a check to see if all the branches have completed - this test fails always, hence workflow is in an indefinite wait state kind of a thing.
      • Join is not executed.

       

      Thanks

      Manju

        • 1. Re: Is the simple fork/join with async execution paths broken in jBPM 4.0
          sebastian.s

          I can't find anything in JIRA but I remember an issue related to fork/joins if I am not mistaken. I consider you to go and grab 4.3 and test if your issues persist. jBPM 4.0 was the first 4.x-release and quite some issues have been addressed in 4.1, 4.2 and 4.3. Furthermore if you are still facing the problem with a more recent version of 4.x chances are higher than somebody else is affected too and can provide some support. There is no reason for using jBPM 4.0 when the most recent one is jBPM 4.3 and everybody's waiting for jBPM 4.4.

          • 2. Re: Is the simple fork/join with async execution paths broken in jBPM 4.0

            I had a similar problem with a simple process running three Java actions in parallel using a fork and a join. The join activity was started three times, once for each incoming transition, but never ended. I used jBPM 4.3 with the job executor enabled. I solved it by specifying that the join should also execute asynchronously. I have not found anything in the documentation about this, but it seems to work.

             

            The fork looked like:

             

            <fork name="Fork">

                <on event="end" continue="exclusive" />

                <transition to="A" />

                <transition to="B" />

                <transition to="C" />

            </fork>

             

            And the join looked like:

             

            <join name="Join" continue="exclusive">

                <transition to="End" />

            </join>

             

            The full example process is attached below.

             

            Regards, Johan

            • 3. Re: Is the simple fork/join with async execution paths broken in jBPM 4.0
              rebody

              Hi Johan,

                I have tested both scenario with exclusive on join and without exclusive on join. And both of test case passed. My environment is jBPM-4.3 hsqldb-1.8.0.10. I uploaded the test project, Somebody who interested about this could have a try.

              • 4. Re: Is the simple fork/join with async execution paths broken in jBPM 4.0

                Hi,

                Thanks for your answer! I have now found the time to try out your example project. Both scenarios worked.

                 

                However, I used the Job Executor to execute jobs, while you execute them programmatically. In my previous test I also had test activities that executed for some time, not just printing a line to stdout. I apologize for not attaching the full source code of my example from the start!

                 

                When I modified your test project to use the Job Executor, and having activities that execute for a while, X0504Exclusive2Test failed in the same way as my own test project. The Join activity is started three times, but the End activity is never started. I attach my modified project with this post.

                 

                One difference you can notice is that when you use the Job Executor, the activities are executed by different threads. When you execute jobs programmatically, the activities are actually executed in sequence by the main thread, even though you have specified in the process that they should be executed in parallel. I attach log files from the two different runs as well.

                 

                Regards, Johan

                • 5. Re: Is the simple fork/join with async execution paths broken in jBPM 4.0
                  swiderski.maciej

                  Hi,

                   

                  make sure that all execution is done exclusive within fork - join boundaries.

                   

                  To make your test pass please add continuation of java activities as exclusive and the same for join

                   

                  <java /class="com.family168.jbpm4book.x01.x05forkjoin.x0504.TestAction" g="146,239,92,52" method="execute" name="A" continue="exclusive">
                  

                   

                   

                  <join g="274,331,48,48" name="Join" continue="exclusive">
                  

                   

                  HTH

                  Maciej

                  • 6. Re: Is the simple fork/join with async execution paths broken in jBPM 4.0

                    Thank you! That confirms the observation I made in my first post, but could not find in the documentation.

                    Regards, Johan

                    • 7. Re: Is the simple fork/join with async execution paths broken in jBPM 4.0
                      swiderski.maciej

                      Hi,

                       

                      ooo, I missed your previous post.

                       

                      When I ran the test it did not went well until I added async continuation to java activities as well.

                       

                      I agree that it would be good to include some info in the guide.

                       

                      Cheers,

                      Maciej

                      • 8. Re: Is the simple fork/join with async execution paths broken in jBPM 4.0
                        rhodos

                        Can anyone please explain why one needs to put async continuations on each activity within a fork in order for things to advance automatically?

                         

                        Thanks,

                        Rachel

                        • 9. Re: Is the simple fork/join with async execution paths broken in jBPM 4.0
                          aguizar
                          Can anyone please explain why one needs to put async continuations on each activity within a fork in order for things to advance automatically?

                          Good question. I've filed JBPM-2888 to investigate this problem. I'm guessing it is related to JBPM-2787: process instances getting stuck in a fork, for jBPM3. There, setting the async attribute in the join node is part of the solution. What strikes me as odd reading the comments in this thread is that every activity following the fork needs to be made asynchronous in jBPM4.

                          • 10. Re: Is the simple fork/join with async execution paths broken in jBPM 4.0
                            aessiari

                            Hello.

                             

                            I had the same problem but I use lockmode="force" for the join activities and it works.

                            My process has asynchronous nested forks, asynchronous java activities, and I loop

                            over these forks up to 100 times.

                             

                            PS: The user guide does not mention force as one of the choices for lockmode.