Version 73
Created by paul.robinson on Jul 20, 2012 9:07 AM. Last modified by paul.robinson on Sep 18, 2012 10:36 AM.

    Introduction

    The purpose of this article is to give you a whirlwind tour of using and configuring XTS. It starts with a simple use case on a single server and then expands to a more typical scenario deployed across multiple servers. Along the way the article will show you what happens when common errors are made and will provide tips on debugging common issues. The article is rather brief in parts but does cover a lot of ground. It is not intended as a replacement for the documentation.

     

    The article was designed to be ran straight through, but you should be able to miss out the sections marked "optional".

     

    Feedback is encouraged, just add a comment to the page!

     

    In particular the article covers:

     

    1. Running a basic single server example

    2. Running the example over multiple servers

    3. Configure Gateway
    4. Common Issues
      1. Endpoint inaccessible
      2. Missing XTS Handlers
      3. AS7 started without XTS
    5. Recovery example
      1. Generate a failure
      2. Inspect the Recovery Log
      3. Observe recovery
    6. Transaction Bridge
      1. Show simple example
      2. Common Issues
        1. Missing bridge handler
        2. Incorrect handler ordering

     

    Pre-Requisites

     

    Create Network Interfaces

    One way of booting multiple AS7 instances on a single machine, is to bind them to different network adapters. As most machines don't have many physical adapters, we will alias the loopback adapter instead:

     

    Fedora:

     

    sudo ifconfig lo:2 127.0.0.2
sudo ifconfig lo:3 127.0.0.3

     

    OSX:

     

    sudo ifconfig lo0 alias 127.0.0.2
sudo ifconfig lo0 alias 127.0.0.3

     

     

    Create Three AS7 Servers

    Download AS7  and then create three separate instances:

     

    wget http://download.jboss.org/jbossas/7.1/jboss-as-7.1.1.Final/jboss-as-7.1.1.Final.zip
unzip ~/Downloads/jboss-as-7.1.1.Final.zip
cp -r jboss-as-7.1.1.Final jboss-as-7.1.1.Final-client
cp -r jboss-as-7.1.1.Final jboss-as-7.1.1.Final-coord
cp -r jboss-as-7.1.1.Final jboss-as-7.1.1.Final-participant

     

    Copy the XTS configuration from the examples folder into the configuration folder:

     

    cp jboss-as-7.1.1.Final-client/docs/examples/configs/standalone-xts.xml jboss-as-7.1.1.Final-client/standalone/configuration/
cp jboss-as-7.1.1.Final-coord/docs/examples/configs/standalone-xts.xml jboss-as-7.1.1.Final-coord/standalone/configuration/
cp jboss-as-7.1.1.Final-participant/docs/examples/configs/standalone-xts.xml jboss-as-7.1.1.Final-participant/standalone/configuration/

     

    Configure the coordinator and participant servers to bind to a different IP address alias:

     

    sed -ie 's/127\.0\.0\.1/127\.0\.0\.2/g'  ./jboss-as-7.1.1.Final-coord/standalone/configuration/standalone-xts.xml 
sed -ie 's/127\.0\.0\.1/127\.0\.0\.3/g'  ./jboss-as-7.1.1.Final-participant/standalone/configuration/standalone-xts.xml

sed -ie 's/127\.0\.0\.1/127\.0\.0\.2/g'  ./jboss-as-7.1.1.Final-coord/standalone/configuration/standalone.xml 
sed -ie 's/127\.0\.0\.1/127\.0\.0\.3/g'  ./jboss-as-7.1.1.Final-participant/standalone/configuration/standalone.xml

     

    Note: We have modified both the configuration needed by XTS (standalone-xts.xml) and the default configuration (standalone.xml). The XTS configuration is used for most of this guide, but we do need to use the default configuration for the section that shows how applications can fail if XTS is not enabled.

    Single Server Scenario

     

    Start the Client instance:

     

    cd jboss-as-7.1.1.Final-client/
sh bin/standalone.sh --server-config=standalone-xts.xml

     

    You can sannity check the startup by first checking that the server started without any errors. A quick way to do this is by inspecting the 'started' log message and ensuring that it doesn't mention that the server started with errors:

    16:58:56,982 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS7.0.GA (AS 7.1.2.Final-redhat-1) started in 20898ms - Started 172 of 257 services (84 services are passive or on-demand)

     

    Another thing to look for is the presence of the XTS Web services. This tells us that the services required by XTS where succesfully installed. You should look for a number of log messages like the following:

    14:58:21,706 INFO  [org.jboss.wsf.stack.cxf.metadata.MetadataBuilder] (MSC service thread 1-5) Add Service
 id=com.arjuna.webservices11.wscoor.sei.ActivationPortTypeImpl
 address=http://localhost:80/ws-c11/ActivationService
 implementor=com.arjuna.webservices11.wscoor.sei.ActivationPortTypeImpl
 invoker=org.jboss.wsf.stack.cxf.JBossWSInvoker
 serviceName={http://docs.oasis-open.org/ws-tx/wscoor/2006/06}ActivationService
 portName={http://docs.oasis-open.org/ws-tx/wscoor/2006/06}ActivationPortType
 wsdlLocation=null
 mtomEnabled=false

     

    From the address value we can see that this service is for the ActivationService. The full list of services are as follows. If you are experiencing problems with a particular service, you should check the corresponding log to ensure it is configured correctly.

    ActivationService
RegistrationService
CompletionInitiatorService
TerminationParticipantService
BusinessAgreementWithCoordinatorCompletionCoordinatorService
BusinessAgreementWithParticipantCompletionCoordinatorService
CompletionCoordinatorRPCService
CompletionCoordinatorService
CoordinatorService
TerminationCoordinatorRPCService
TerminationCoordinatorService
BusinessAgreementWithCoordinatorCompletionParticipantService
BusinessAgreementWithParticipantCompletionParticipantService
ParticipantService

     

     

    Now obtain the AS7 Quickstarts from http://github.com/jboss-jdf/jboss-as-quickstart/zipball/7.1.1.Final and extract, with the following command:

     

    unzip 7.1.1.Final

     

    Run the wsat-simple quickstart:

     

    cd jboss-jdf-jboss-as-quickstart-8505020/wsat-simple/
mvn clean test -Parq-jbossas-remote

     

    You should see:

     

    Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16.795s
[INFO] Finished at: Fri Jul 20 15:25:10 BST 2012
[INFO] Final Memory: 17M/81M
[INFO] ------------------------------------------------------------------------

     

    See the README.md in the wsat-simple quickstart for more details on the quickstart and the expected output.

     

     

    Multiple Server Scenario

     

    First we'll make use of an external coordinator. We will be using the coordinator deployed to the second server instance which we configured earlier to bind to '127.0.0.1'. Configure this by editing jboss-as-7.1.1.Final-client/standalone/configuration/standalone-xts.xml and changing:

             <subsystem xmlns="urn:jboss:domain:xts:1.0">
            <xts-environment url="http://${jboss.bind.address:127.0.0.1}:8080/ws-c11/ActivationService"/>
        </subsystem>

    To:

     

             <subsystem xmlns="urn:jboss:domain:xts:1.0">
            <xts-environment url="http://127.0.0.2:8080/ws-c11/ActivationService"/>
        </subsystem>

     

    Next we need to deploy the Restaurant Service to the third server instance. Do this by running the following from the wsat-simple project directory:

     

    mvn install
cp target/jboss-as-wsat-simple.war ../../jboss-as-7.1.1.Final-participant/standalone/deployments/

     

    Now update the wsat-simple test to use the remote service. Do this by editing ./src/test/java/org/jboss/as/quickstarts/wsat/simple/Client.java and set

     

     URL wsdlLocation = new URL("http://localhost:8080/wsat-simple/RestaurantServiceAT?wsdl");

    To:

    URL wsdlLocation = new URL("http://127.0.0.3:8080/jboss-as-wsat-simple/RestaurantServiceAT?wsdl");

     

    Now start the second and third servers:

    cd jboss-as-7.1.1.Final-coord/
sh bin/standalone.sh --server-config=standalone-xts.xml

cd jboss-as-7.1.1.Final-participant/
sh bin/standalone.sh --server-config=standalone-xts.xml

     

     

    Check they start without error and then run the test:

    cd jboss-jdf-jboss-as-quickstart-8505020/wsat-simple/
mvn clean test -Parq-jbossas-remote

     

    You should see:

    Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16.795s
[INFO] Finished at: Fri Jul 20 15:25:10 BST 2012
[INFO] Final Memory: 17M/81M
[INFO] ------------------------------------------------------------------------

     

    You should now take a look at each of the server logs. The first server (client) should have a number of logs containing '[CLIENT]'. The second server (coordinator) should have logs for each of the created services. The third server (participant) shuld have a number of logs containing '[SERVICE]'

     

    Configure Gateway (Optional)

    In this section we will put an apache server in front of the participant AS7 instance. We will use Apache with mod_proxy to act as a gateway, forwarding all http requests from  localhost:80 to 127.0.0.3:8080 (the participant server). It is assumed that you already have Apache httpd installed.

     

    Configure the proxy, by editing your httpd.conf file. This is located at '/etc/apache2/httpd.conf' on a Mac. Add the following to the bottom of the file:

     

    ProxyRequests Off
ProxyPreserveHost On

<Proxy *>
        Order deny,allow
        Allow from all
</Proxy>

ProxyPass / http://127.0.0.3:8080/
ProxyPassReverse / http://127.0.0.3:8080/
<Location />
        Order allow,deny
        Allow from all
</Location>

    Note: These are just simple settings used for this example. You should consider carefully your own requirements and configure accordingly before using this in production.

     

    Now restart Apache:

    sudo apachectl restart

     

    You should now be able to access the WSDL through the gateway at localhost:80. Run the following command to observe the WSDL:

    curl  http://localhost/jboss-as-wsat-simple/RestaurantServiceAT?wsdl

     

    However, if you take a look at the 'soap:address' element, you will see that it is still advertising the direct address of the service, rather than address accessible through the proxy:

    <soap:address location="http://127.0.0.3:8080/jboss-as-wsat-simple/RestaurantServiceAT"/>

     

    To fix this we need to stop the participant AS7 instance and then edit standalone/configuration/standalone-xts.xml. Change:

    <wsdl-host>${jboss.bind.address:127.0.0.3}</wsdl-host>

    To:

    <wsdl-host>localhost</wsdl-host>
<wsdl-port>80</wsdl-port>

    Then start the server back up again.

     

    This configuration change will ensure that the 'soap:address' element of the served WSDL documents are re-written to advertise the service location at "http://localhost:80/...". If you re-visit http://localhost/jboss-as-wsat-simple/RestaurantServiceAT?wsdl you should  see that the soap:address is now:

    <soap:address location="http://localhost:80/jboss-as-wsat-simple/RestaurantServiceAT"/>

     

    This change also applies to the XTS Web services. You can check this by looking up the XTS Web service details in the log and then checking the WSDL as we did earlier in this guide. Here's an example for one of the services:

    14:58:21,706 INFO  [org.jboss.wsf.stack.cxf.metadata.MetadataBuilder] (MSC service thread 1-5) Add Service
 id=com.arjuna.webservices11.wscoor.sei.ActivationPortTypeImpl
 address=http://localhost:80/ws-c11/ActivationService
 implementor=com.arjuna.webservices11.wscoor.sei.ActivationPortTypeImpl
 invoker=org.jboss.wsf.stack.cxf.JBossWSInvoker
 serviceName={http://docs.oasis-open.org/ws-tx/wscoor/2006/06}ActivationService
 portName={http://docs.oasis-open.org/ws-tx/wscoor/2006/06}ActivationPortType
 wsdlLocation=null
 mtomEnabled=false

     

    Here you can see the address is set to 'http://localhost:80/ws-c11/ActivationService' and then if you view http://localhost:80/ws-c11/ActivationService?wsdl you should see the soap:address is set to:

    <soap:address location="http://localhost:80/ws-c11/ActivationService"/>

     

    We can now re-run the test, accessing the participant service through the Apache server. Before we do this we will need to edit 'jboss-jdf-jboss-as-quickstart-8505020/wsat-simple/src/test/java/org/jboss/as/quickstarts/wsat/simple/Client.java' and change the URL as follows:

    URL wsdlLocation = new URL("http://localhost/jboss-as-wsat-simple/RestaurantServiceAT?wsdl");

     

    You can now re-run the tests and observe that they pass.

     

    Common Miss-configuration Issues (Optional)

    Endpoint inaccessible (Optional)

    Sometimes you get problems where during a WS-AT or WS-BA, an endpoint can't be contacted. These types of problems can occur if you have missconfigured your network or gateway or if a server is down. To see this in action, lets shutdown the second server (coordinator) and re-run the tests.

     

    The Activation Service, hosted by the coordinator can no longer be accessed. You will see the following Exception thrown from the client:

    ConnectException invoking http://127.0.0.2:8080/ws-c11/ActivationService: Connection refused

     

    A good place to start when debugging these issues is to check that the expected SOAP endpoint exists. When XTS starts a number of Web services should be deployed. JBossWS logs the details of each of these Web services when AS7 boots. For example, this is logged when the XTS coordinator is deployed:

    17:05:09,224 INFO  [org.jboss.wsf.stack.cxf.metadata.MetadataBuilder] (MSC service thread 1-14) Add Service
     id=com.arjuna.webservices11.wscoor.sei.ActivationPortTypeImpl
     address=http://127.0.0.2:8080/ws-c11/ActivationService
     implementor=com.arjuna.webservices11.wscoor.sei.ActivationPortTypeImpl
     invoker=org.jboss.wsf.stack.cxf.JBossWSInvoker
     serviceName={http://docs.oasis-open.org/ws-tx/wscoor/2006/06}ActivationService
     portName={http://docs.oasis-open.org/ws-tx/wscoor/2006/06}ActivationPortType
     wsdlLocation=null
     mtomEnabled=false

     

    You can check that the endpoint is serving up WSDL by following the WSDL url and checking a valid document is returned. To create this URL, take the 'address' value and append '?wsdl'. For example, you should be able to visit http://127.0.0.2:8080/ws-c11/ActivationService?wsdl. You should check that these addresses are contactable from the host that is having problems connecting, so lets run the following commands from the machine running the client:

    curl http://127.0.0.2:8080/ws-c11/ActivationService?wsdl

     

    You should see the following error (or similar) as the server was not available:

    curl: (7) couldn't connect to host

     

    Now start the second server (coordinator) back up and try the above command again. Youe should see the wsdl document dumped to your terminal.

     

    Sometimes you hit issues where the wsdl is accessible, but it advertises an invalid address for the Web service endpoint. To check this, take a look at the 'soap:address' element, of the WSDL, and check that the location is as expected:

    <soap:address location="http://127.0.0.2:8080/ws-c11/ActivationService"/>

     

    Missing XTS Handlers (Optional)

    The developer is required to specify SOAP handlers on the client and servers side SOAP stack. This is the integration point for XTS. If these are missing, transaction propagation will not occur. In this section we'll see what happens when you omit them.

     

    Missing Client-side Handler (Optional)

    Remove the client-side handler by editing ./src/test/java/org/jboss/as/quickstarts/wsat/simple/Client.java and changing:

    /*
 * Add client handler chain
 */
BindingProvider bindingProvider = (BindingProvider) restaurant;
List<Handler> handlers = new ArrayList<Handler>(1);
handlers.add(new JaxWSHeaderContextProcessor());
bindingProvider.getBinding().setHandlerChain(handlers);

    To:

    /*
 * Add client handler chain
 */
//BindingProvider bindingProvider = (BindingProvider) restaurant;
//List<Handler> handlers = new ArrayList<Handler>(1);
//handlers.add(new JaxWSHeaderContextProcessor());
//bindingProvider.getBinding().setHandlerChain(handlers);

     

    By doing this you have removed the code that configures the client-side XTS handler.

     

    Now re-run the test and observe two failures:

    mvn test -Parq-jbossas-remote

     

    From looking at the participant server log, you will see:

    Caused by: com.arjuna.wst.UnknownTransactionException
        at com.arjuna.mwlabs.wst11.at.remote.TransactionManagerImple.enlistForDurableTwoPhase(TransactionManagerImple.java:56) [jbossxts-4.16.2.Final.jar:]
        at org.jboss.as.quickstarts.wsat.simple.RestaurantServiceATImpl.makeBooking(RestaurantServiceATImpl.java:70)

     

     

    To understand what is happening, you need to look at  line 70 of ./src/main/java/org/jboss/as/quickstarts/wsat/simple/RestaurantServiceATImpl.java:

    transactionManager.enlistForDurableTwoPhase(restaurantParticipant, "restaurantServiceAT:" + UUID.randomUUID());

     

    As the XTS handler was missing on the client-side, no transaction context was placed in the SOAP header of the message. As a result the XTS server side handler chain assumed that the request was not part of a transaction. When 'transactionManager.enlistForDurableTwoPhase(...)' is invoked, the 'UnknownTransactionException' is thrown due to the current transaction not being known.

     

    NOTE: Remember to un-comment the code for adding the XTS handler in ./src/test/java/org/jboss/as/quickstarts/wsat/simple/Client.java.

     

    Missing Server-side Handler (Optional)

    Edit './src/main/java/org/jboss/as/quickstarts/wsat/simple/RestaurantServiceATImpl.java' and comment out the following line. This line is responsible for defining what SOAP handler chain should be used for this Web service. By commenting out the line you are preventing the XTS inbound handler from being added.

    @HandlerChain(file = "/context-handlers.xml", name = "Context Handlers")

     

    Now re-build and deploy the participant service and re-run the tests. You should then observe two failures.

    mvn install
    cp target/jboss-as-wsat-simple.war ../../jboss-as-7.1.1.Final-participant/standalone/deployments/
    mvn test -Parq-jbossas-remote

     

    If you look on the participant AS log, you wil see the following stack trace:

    15:56:19,372 WARNING [org.apache.cxf.phase.PhaseInterceptorChain] (http--127.0.0.3-8080-1) Interceptor for {http://www.jboss.com/jbossas/quickstarts/wsat/simple/Restaurant}RestaurantServiceATService#{http://www.jboss.com/jbossas/quickstarts/wsat/simple/Restaurant}makeBooking 
has thrown exception, unwinding now: org.apache.cxf.binding.soap.SoapFault: MustUnderstand headers: [{http://docs.oasis-open.org/ws-tx/wscoor/2006/06}CoordinationContext] are not understood.

     

    Here you can see that the incoming SOAP message had a CoordinationContext header that is marked as 'MustUnderstand'. As we removed the XTS server side handler, there was no handler that understood this header. Therefore this exception is thrown by the SOAP stack when processing the incoming message.

     

    NOTE: Remember to un-comment the code for adding the XTS handler in ./src/main/java/org/jboss/as/quickstarts/wsat/simple/RestaurantServiceATImpl.java and re-build and deploy the participant service, with the following commands:

    mvn install
cp target/jboss-as-wsat-simple.war ../../jboss-as-7.1.1.Final-participant/standalone/deployments/
mvn test -Parq-jbossas-remote

    No XTS Subsystem running (Optional)

    As XTS is not enabled in the default configuration, it is a common error to try to use XTS when the XTS subsystem is not running. In this section we'll show what can happen if you atempt to run the Client, Coordinator or Participant without XTS enabled.

     

    No XTS for Client (Optional)

    Stop the client instance and then restart it with the default configuration:

    cd jboss-as-7.1.1.Final-client/
sh bin/standalone.sh

     

    Now re-run the wsat-simple test and observe that two failures occur.

    mvn test -Parq-jbossas-remote

     

    As we have a test failure, the first point of call should be the test output. Open up target/surefire-reports/org.jboss.as.quickstarts.wsat.simple.ClientTest.txt and observe the error message. The line of interest is:

    java.lang.NullPointerException
        at org.jboss.as.quickstarts.wsat.simple.ClientTest.testCommit(ClientTest.java:89)

     

    Here we can see that there was a NPE thrown on Line 89 of src/test/java/org/jboss/as/quickstarts/wsat/simple/ClientTest.java. This occurs because the XTS service is unavailable and so the call to 'UserTransactionFactory.userTransaction();' returned null.

     

    NOTE: Remember to restart the client instance with the XTS configuration.

     

    No XTS for Coordinator

    Restart the coordinator with the default configuration:

     

    cd jboss-as-7.1.1.Final-client/
sh bin/standalone.sh --server-config=standalone-xts.xml

cd jboss-as-7.1.1.Final-coord/
sh bin/standalone.sh

     

    Now re-run the wsat-simple test and observe that two failures occur.

    mvn test -Parq-jbossas-remote

     

    If we take a look at the client test log (target/surefire-reports/org.jboss.as.quickstarts.wsat.simple.ClientTest.txt) we see the following stack trace:

    com.arjuna.wst.SystemException: javax.xml.ws.WebServiceException: Could not send Message.
        at com.arjuna.mwlabs.wst11.at.remote.UserTransactionImple.startTransaction(UserTransactionImple.java:308)
        at com.arjuna.mwlabs.wst11.at.remote.UserTransactionImple.begin(UserTransactionImple.java:80)
        at com.arjuna.mwlabs.wst11.at.remote.UserTransactionImple.begin(UserTransactionImple.java:70)
        at org.jboss.as.quickstarts.wsat.simple.ClientTest.testCommit(ClientTest.java:89)

     

    This tells us that XTS failed to send a Web service message when atempting to start the transaction. This message doesn't really tell us enough as we don't know what address was trying to be contacted. If we have a look at the client AS7 server log we see this stack trace:

    Caused by: org.apache.cxf.transport.http.HTTPException: HTTP response '404: Not Found' when communicating with http://127.0.0.2:8080/ws-c11/ActivationService

     

    As we got a 404 response, we know the server is running, but it looks like there is no service at '/ws-c11/ActivationService'. By looking at the server log of the coordinator you should notice that no endpoint was started for the ActivationService. You know this as the following output is *NOT* present:

    17:05:09,224 INFO  [org.jboss.wsf.stack.cxf.metadata.MetadataBuilder] (MSC service thread 1-14) Add Service
 id=com.arjuna.webservices11.wscoor.sei.ActivationPortTypeImpl
 address=http://127.0.0.2:8080/ws-c11/ActivationService
 implementor=com.arjuna.webservices11.wscoor.sei.ActivationPortTypeImpl
 invoker=org.jboss.wsf.stack.cxf.JBossWSInvoker
 serviceName={http://docs.oasis-open.org/ws-tx/wscoor/2006/06}ActivationService
 portName={http://docs.oasis-open.org/ws-tx/wscoor/2006/06}ActivationPortType
 wsdlLocation=null
 mtomEnabled=false

     

    NOTE: Remember to restart the coordinator instance with the XTS configuration.

    No XTS For Participant

    Restart the participant with the default configuration:

     

    cd jboss-as-7.1.1.Final-coord/
    sh bin/standalone.sh --server-config=standalone-xts.xml
     
    cd jboss-as-7.1.1.Final-participant/
    sh bin/standalone.sh

     

    Now re-run the wsat-simple test and observe that two failures occur.

    mvn test -Parq-jbossas-remote

     

    If you take a look at the client test log, client server log and coordinator server log, you'll see a number of stack traces. It's not immediately obvious, from these, what has gone wrong. However, if you take a look at the participant server log, you will see:

    Caused by: java.lang.NullPointerException
        at org.jboss.as.quickstarts.wsat.simple.RestaurantServiceATImpl.makeBooking(RestaurantServiceATImpl.java:64) [classes:]

     

    And then by looking at line 64 of ./src/main/java/org/jboss/as/quickstarts/wsat/simple/RestaurantServiceATImpl.java you should see:

    transactionId = UserTransactionFactory.userTransaction().toString();

     

    What happened was that 'UserTransactionFactory.userTransaction()' returned null as the XTS service was not available. This then caused the NPE when 'toString()' was invoked.

     

    Note: remember to restart the participant AS7 instance using the standalone-xts.xml configuration.

     

    Recovery (Optional)

    For this example you will need to obtain the wsat-recovery quickstart. This quickstart is in the early stages of development but should serve us well to demonstrate the basics of recovery.

     

    git clone git://github.com/paulrobinson/quickstart.git
cd quickstart
git checkout remotes/origin/wsat-recovery
cd wsat-recovery

     

    Now remove the existing participant:

    rm /opt/xts-demo-eap/jboss-as-7.1.1.Final-participant/standalone/deployments/jboss-as-wsat-simple.*

     

    Build and deploy the wsat-recovery participant:

    mvn install
cp target/jboss-as-wsat-recovery.war /opt/xts-demo-eap/jboss-as-7.1.1.Final-participant/standalone/deployments/

     

    Generate a Failure

    Start the participant AS configured to crash the participant during commit:

    sh bin/standalone.sh --server-config=standalone-xts.xml -Dcrash=true

     

    Run the wsat-recovery test:

    mvn test -Parq-jbossas-remote

     

    You should observe that the test fails. This is because the participant server crashed during commit and a SystemException was returned to the client which causes the test to be marked as a failure.

    Results :

Tests in error: 
  testCommit(org.jboss.as.quickstarts.wsat.recovery.ClientTest)
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

     

    If you look at the participant server's console you should notice that the server has crashed with its dying words visible on the console:

    515:41:38,247 INFO  [stdout] (http--127.0.0.3-8080-1) [SERVICE] Restaurant service invoked to make a booking
15:41:38,288 INFO  [stdout] (http--127.0.0.3-8080-1) [SERVICE] Enlisting a Durable2PC participant into the AT
15:41:38,367 INFO  [org.apache.cxf.service.factory.ReflectionServiceFactoryBean] (http--127.0.0.3-8080-1) Creating Service {http://docs.oasis-open.org/ws-tx/wscoor/2006/06}RegistrationService from WSDL: jar:file:/opt/xts-demo-eap/jboss-as-7.1.1.Final-participant/modules/org/jboss/xts/main/jbossxts-4.16.2.Final.jar!/org/oasis_open/docs/ws_tx/wscoor/_2006/_06/wsdl/wscoor-registration-binding.wsdl
15:41:38,382 INFO  [org.jboss.wsf.stack.cxf.transport.AddressRewritingEndpointInfo] (http--127.0.0.3-8080-1) Setting new service endpoint address in wsdl: http://localhost:9000/ws-c11/RegistrationService
15:41:38,464 INFO  [stdout] (http--127.0.0.3-8080-1) [SERVICE] Invoking the back-end business logic
15:41:38,464 INFO  [stdout] (http--127.0.0.3-8080-1) [SERVICE] makeBooking called on backend resource.
15:41:39,082 INFO  [stdout] (TaskWorker-1) [SERVICE] Prepare called on participant, about to prepare the back-end resource
15:41:39,083 INFO  [stdout] (TaskWorker-1) [SERVICE] prepare called on backend resource.
15:41:39,083 INFO  [stdout] (TaskWorker-1) [SERVICE] back-end resource prepared, participant votes prepared
15:41:39,222 INFO  [org.apache.cxf.service.factory.ReflectionServiceFactoryBean] (TaskWorker-1) Creating Service {http://docs.oasis-open.org/ws-tx/wsat/2006/06}CoordinatorService from WSDL: jar:file:/opt/xts-demo-eap/jboss-as-7.1.1.Final-participant/modules/org/jboss/xts/main/jbossxts-4.16.2.Final.jar!/org/oasis_open/docs/ws_tx/wsat/_2006/_06/wsdl/wsat-coordinator-binding.wsdl
15:41:39,278 INFO  [stdout] (TaskWorker-1) [SERVICE] Inside commit, about to crash service...

     

    Inspect the Recovery Log

    Before starting up the server and observing recovery, we should take a look at the recovery log. First of all we'll inspect the tx-object-store directory on the participant server. Here you can see that there is one ParticipantRecoveryRecord:

    # tree standalone/data/tx-object-store

standalone/data/tx-object-store
└── ShadowNoFileLockStore
    └── defaultStore
        └── XTS
            └── WSAT
                └── ParticipantRecoveryRecord
                    └── 0_ffffac11820b_66a0f450_5018e09c_13

5 directories, 1 file

     

    Now we can take a look at this record by opening the file in a text editor. Outputting the contents of the file, to the console, can confuse the console due to the binary data present in the file, so opening in a text editor is reccomended.

    # vi standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/XTS/WSAT/ParticipantRecoveryRecord/0_ffffac11820b_66a0f450_5018e09c_13

    We can also observe that the coordinator has a recovery record logged for this transaction. This ensures that if it crashes it can still commit the transaction:

     

    # tree /opt/xts-demo-eap/jboss-as-7.1.1.Final-coord/standalone/data/tx-object-store/
/opt/xts-demo-eap/jboss-as-7.1.1.Final-coord/standalone/data/tx-object-store/
└── ShadowNoFileLockStore
    └── defaultStore
        └── StateManager
            └── BasicAction
                └── AtomicAction
                    └── TwoPhaseCoordinator
                        └── TwoPhase
                            └── ATCoordinator
                                └── 0_ffffac11820b_32091c0b_501ab815_52

8 directories, 1 file

     

    Again, you can view the contents with with a command similar to:

     

    vi /opt/xts-demo-eap/jboss-as-7.1.1.Final-coord/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/AtomicAction/TwoPhaseCoordinator/TwoPhase/ATCoordinator/0_ffffac11820b_32091c0b_501ab815_52
    Observe Recovery

    Now we should start the server back up in order to recover the transaction. This time we'll omit the 'crash' property as we don't want the server to crash when the commit message it is replayed.

    sh bin/standalone.sh --server-config=standalone-xts.xml

     

    When the server boots, you will see the following log message. This line is output when the application registers the recovery module:

    08:37:08,258 INFO  [stdout] (MSC service thread 1-10) [RECOVERY] Registering recovery module

     

    You should also see many lines like the following start to appear. These appear because the coordinator is continually replaying the commit message (every 5s) that it is yet to receive a valid response for. 

    08:37:09,614 WARN  [com.arjuna.wst] (TaskWorker-1) ARJUNA043159: Commit request dropped pending WS-AT participant recovery manager scan for unknown participant: restaurantServiceAT:2d77d068-9500-402a-af4c-7d84edb3d67e

     

    You can confirm this by looking at the coordinator log. Here you will see corresponding log entries, similar to the following at 5s intervals, matching the time of the "Commit request dropped..." messages above.

    08:37:09,341 INFO  [org.apache.cxf.service.factory.ReflectionServiceFactoryBean] (Timer-1) Creating Service {http://docs.oasis-open.org/ws-tx/wsat/2006/06}ParticipantService from WSDL: jar:file:/opt/xts-demo-eap/jboss-as-7.1.1.Final-coord/mod
ules/org/jboss/xts/main/jbossxts-4.16.2.Final.jar!/org/oasis_open/docs/ws_tx/wsat/_2006/_06/wsdl/wsat-participant-binding.wsdl

     

    When recovery occurs, you should see log entries similar to the following. 

    08:39:26,581 INFO  [stdout] (Periodic Recovery) [RECOVERY] Attempting to deserialize RestaurantParticipant restaurantServiceAT:2d77d068-9500-402a-af4c-7d84edb3d67e
08:39:26,586 INFO  [stdout] (Periodic Recovery) [RECOVERY] Deserialized RestaurantParticipantAT restaurantServiceAT:2d77d068-9500-402a-af4c-7d84edb3d67e
08:39:26,674 INFO  [org.apache.cxf.service.factory.ReflectionServiceFactoryBean] (Periodic Recovery) Creating Service {http://docs.oasis-open.org/ws-tx/wsat/2006/06}CoordinatorService from WSDL: jar:file:/opt/xts-demo-eap/jboss-as-7.1.1.Final-participant/modules/org/jboss/xts/main/jbossxts-4.16.2.Final.jar!/org/oasis_open/docs/ws_tx/wsat/_2006/_06/wsdl/wsat-coordinator-binding.wsdl
08:39:26,770 INFO  [stdout] (Periodic Recovery) [RECOVERY] First scan complete. Backend resource should tidy up any orphaned transactions.
08:39:26,783 INFO  [stdout] (TaskWorker-1) [SERVICE] Not crashing the service
08:39:26,784 INFO  [stdout] (TaskWorker-1) [SERVICE] all participants voted 'prepared', so coordinator tells the participant to commit
08:39:26,784 INFO  [stdout] (TaskWorker-1) [SERVICE] Committing booking for name 'Paul'
08:39:26,785 INFO  [stdout] (TaskWorker-1) [SERVICE] commit called on backend resource.

     

    Here you can see that the recovery module was invoked, and the participant deserialized. The transaction is then continued, by replaying the commit on the participant. The coordinator then recieves a 'committed' message and stops replaying the 'commit' message. You can confirm this by looking at the coordinator log and observing that the log messages, described above, have stopped appearing.

     

    Transaction Bridge

    This section will show you how to quickly get an example up and running, that demonstrates the bridging of a WS-AT transaction to a JTA transaction. The benefit of using the TXBridge is that the application can re-use existing XA aware resources, such as a database or message service.

     

    Simple Example

    We'll start by running a simple quickstart.

     

    First obtain the quickstarts:

    git clone git://github.com/jbosstm/quickstart.git narayana-quickstart
cd narayana-quickstart/TXBridge/demo/

     

    Now modify the location of the service. We will be deploying the service to our participant server instance. Therefore we must change the client to lookup the wsdl from that location. To do this edit ./client/src/main/java/org/jboss/jbossts/txbridge/demo/client/BasicClient.java and change:

     

    URL wsdlLocation = new URL("http://localhost:8080/txbridge-demo-service/BistroImpl?wsdl");

    To:

    URL wsdlLocation = new URL("http://127.0.0.1/txbridge-demo-service/BistroImpl?wsdl");

     

    NOTE: If you didn't setup the gateway, you will need to change the address to "http://127.0.0.3:8080/txbridge-demo-service/BistroImpl?wsdl"

     

    Now build and deploy the application:

    mvn install -DskipTests=true
cp client/target/txbridge-demo-client.war /opt/xts-demo-eap/jboss-as-7.1.1.Final-client/standalone/deployments/
cp service/target/txbridge-demo-service.jar /opt/xts-demo-eap/jboss-as-7.1.1.Final-participant/standalone/deployments/

     

    Now visit http://127.0.0.1:8080/txbridge-demo-client/ and then click on "Submit Booking". You should see the page re-load and "Transaction Result: Transaction finished OK." appear under the title. If you look at the client log you can see some output that shows the progress of the transaction, from the point of view of the client. You won't see much useful output on the participant server as the participant is an XA Resource, which you wouldn't expect to see too much logging from.

     

    Missing Bridge Handler (Optional)

    XTS bridges the incoming WS-AT transaction onto a JTA transaction at the point when the application's SOAP message enters the server. It does this through SOAP handlers. In this section we will investigate what happens if the TXBridge handler is omitted from the handler chain.

     

    First remove the TXBridge handler from the handler chain configuration, by editing service/src/main/resources/org/jboss/jbossts/txbridge/demo/bistro/jaxws-handlers-server.xml and changing:

     

    <handler>
    <handler-name>TransactionBridgeHandler</handler-name>
    <handler-class>org.jboss.jbossts.txbridge.inbound.JaxWSTxInboundBridgeHandler</handler-class>
</handler>

    To:

    <!--handler>
    <handler-name>TransactionBridgeHandler</handler-name>
    <handler-class>org.jboss.jbossts.txbridge.inbound.JaxWSTxInboundBridgeHandler</handler-class>
</handler-->

     

    Now rebuild and re-deploy the service:

    mvn install -DskipTests=true
cp service/target/txbridge-demo-service.jar /opt/xts-demo-eap/jboss-as-7.1.1.Final-participant/standalone/deployments/

     

    Now re-visit http://127.0.0.1:8080/txbridge-demo-client/basicclienthttp://127.0.0.1:8080/txbridge-demo-client and click "Submit Booking". You should notice a failure this time:

    Transaction Result: Transaction failed! Cause: javax.xml.ws.soap.SOAPFaultException: JBAS014162: Transaction is required for invocation org.jboss.invocation.InterceptorContext@1d4c208

     

    This SOAP fault is returned, due to an EJB exception that is thrown when attempting to invoke the Bistro service. If you open service/src/main/java/org/jboss/jbossts/txbridge/demo/bistro/BistroImpl.java you will notice on, or around, line 46:

    @TransactionAttribute(TransactionAttributeType.MANDATORY) // default is REQUIRED

     

    This annotation says that there must be an existing JTA transaction running when the methods of this service is invoked. Because we removed the TXBridge handler, no JTA transaction was started. There will be a WS-AT transaction associated with this thread and active at the time when this method is invoked. However, the EJB requires a JTA transaction and knows nothing of WS-AT.

     

    What if the developer hadn't guarded for such situations? Lets remove the @TransactionAttribute and see what happens.

     

    Open service/src/main/java/org/jboss/jbossts/txbridge/demo/bistro/BistroImpl.java and change:

    @TransactionAttribute(TransactionAttributeType.MANDATORY) // default is REQUIRED

    To:

    //@TransactionAttribute(TransactionAttributeType.MANDATORY) // default is REQUIRED

     

    Then rebuild and re-deploy the service:

    mvn install -DskipTests=true
    cp service/target/txbridge-demo-service.jar /opt/xts-demo-eap/jboss-as-7.1.1.Final-participant/standalone/deployments/

     

    Now re-visit http://127.0.0.1:8080/txbridge-demo-client and click "Submit Booking", Again you will notice a failure:

    Transaction Result: Transaction failed! Cause: com.arjuna.wst.WrongStateException

     

    This time we get a WrongStateException. If you look at the client server console you will see:

    com.arjuna.wst.WrongStateException
   at com.arjuna.mwlabs.wst11.at.remote.UserTransactionImple.begin(UserTransactionImple.java:78)
   at com.arjuna.mwlabs.wst11.at.remote.UserTransactionImple.begin(UserTransactionImple.java:70)
   at org.jboss.jbossts.txbridge.demo.client.BasicClient.testAtomicTransaction(BasicClient.java:166)

     

    This tells us that the client application is trying to invoke begin when the current transaction is not in a state where it can be begun. But why would you get this exception here? This is the first call to begin a transaction. Or is it? The reason this happens is that the same thread is being used to service this request as was being used to service the previous request. In the previous request we began a transaction, this was associated with the current thread, then a failure occurred. If you look at ./client/src/main/java/org/jboss/jbossts/txbridge/demo/client/BasicClient.java you will see that if an exception is thrown from testAtomicTransaction, the ut.commit() line will not be invoked. This leaves a transaction associated with the thread until commit or rollback is invoked. Thus when the thread is re-used, the call to ut.begin() fails as a transaction is still associated with the thread. This is a bug in the quickstart which we should fix.

     

    IMPORTANT: You now need to reboot the client server instance.

     

    We are back on track now, so we can get back to seeing what happens now that we have removed the TXBridge handler and also the TransactionalAttribute. Re-visit http://127.0.0.1:8080/txbridge-demo-client and click "Submit Booking". You should notice a success this time. So why did this test succeed, when we removed the TXBridge handler? What happened is that we now have two separate, un-related, transactions. The JTA transaction was started when the bookSeats method was invoked and was then committed shortly after the method completed. At this point the WS-AT transaction hadn't even prepared, so there is a chance it may not be committed. Also, a failure just after the commit of the JTA transaction will result in a rollback of the WS-AT transaction, leaving the application in an inconsistent state.

     

    IMPORTANT: The point to take away from this is that it is very easy to make a small configuration change that completely changes the transactional guarantees of the application!

     

    NOTE: remember to un-comment the handler chain configuration and the TransactionalAttribute!

    Incorrect Handler Ordering (Optional)

    The ordering of the XTS and TXBridge handler is important. This is because the TXBridge handler uses thread association for discovering which WS-AT transaction to bridge from. It is easy to get this order wrong as the handlers chain is invoked in reverse order to that which is specified in the configuration file.

     

    To observe this error lets edit service/src/main/resources/org/jboss/jbossts/txbridge/demo/bistro/jaxws-handlers-server.xml and change:

     

    <handler>
    <handler-name>TransactionBridgeHandler</handler-name>
    <handler-class>org.jboss.jbossts.txbridge.inbound.JaxWSTxInboundBridgeHandler</handler-class>
</handler>

<handler>
    <handler-name>WebServicesTxContextHandler</handler-name>
    <handler-class>com.arjuna.mw.wst11.service.JaxWSHeaderContextProcessor</handler-class>
</handler>

    To:

    <handler>    <handler-name>WebServicesTxContextHandler</handler-name>
    <handler-class>com.arjuna.mw.wst11.service.JaxWSHeaderContextProcessor</handler-class>
</handler>

<handler>
    <handler-name>TransactionBridgeHandler</handler-name>
    <handler-class>org.jboss.jbossts.txbridge.inbound.JaxWSTxInboundBridgeHandler</handler-class>
</handler>

     

    Now rebuild and re-deploy the service:

    mvn install -DskipTests=true
    cp service/target/txbridge-demo-service.jar /opt/xts-demo-eap/jboss-as-7.1.1.Final-participant/standalone/deployments/

     

    Now re-visit http://127.0.0.1:8080/txbridge-demo-client and click "Submit Booking". You should notice a failure this time:

    Transaction Result: Transaction failed! Cause: java.lang.NullPointerException

     

    This doesn't tell us too much, but if we look at the participant console we see:

    09:26:19,676 ERROR [org.jboss.jbossts.txbridge] (http-127.0.0.3/127.0.0.3:8080-2) com.arjuna.wst.UnknownTransactionException

     

    From examples, earlier in this document, we know that this exception occurs when we try to enlist a participant when no WS-AT transaction is associated with the current thread. This occurs because the bridge handler is being invoked before the XTS handler. Therefor the XTS handler has not yet had chance to import the WS-AT transaction and create the thread association.