Fault Injection Testing With Byteman

Let's assume you know what Byteman is . . .

 

This is the Part Two of the Byteman tutorial series, explaining how you can use Byteman to improve the simplicity, quality and coverage of your unit, integration and system tests.  Part One is a basic How-To which tells you what Byteman is and how to download and install it and provides simple examples of how to use Byteman to inject side-effects into Java programs. If you are new to Byteman then you probably ought to read part one before you continue with this article. If you already know what Byteman does then this article will help you take the next step, employing it to make your tests simpler and clearer and extending your tests suite to cover test  scenarios which would be difficult or, in some cases, impossible to implement without the use of a tool like Byteman.

 

As with the previous article the material here is structured as a FAQ comprising answers to a series of questions. If you have not used Byteman as a testing tool then you should probably read through from start to finish trying out the examples. If you have written tests with Byteman already and just want to refresh your memory or answer a specific question, you can jump ahead using one of these links. However, in the latter case you might find it helpful to skim the preceding section(s) in order to fully grasp what is being presented.

 


What is Fault Injection Testing?

 

Fault injection is a technique which can be used to make tests, simpler, clearer and more reliable. It involves changing the application code under test at one or more specific locations so that additional code (or, possibly, alternative code) is executed when control reaches that point during the test. By carefully selecting the side effects you introduce and by including appropriate conditions to control when they are run it is possible to simulate complex test scenarios for unusual or erroneous situations, forcing the test  along the code paths meant to handle these abnormal cases. This is particularly useful when trying to test code which has timing dependencies, particularly multi-threaded code.

 

One advantage of fault injection is that it usually requires less effort than alternative forms of testing such as writing test harnesses or implementing mock components. It often takes only a few lines of injected code to ensure that the required test scenario is exercised. A beneficial consequence of this is that the test run mostly exercises real application code rather than code introduced just for the purpose of running the test. This limits the opportunities for the test code to introduce or mask errors or to affect memory footprint and timing. The use of fault injection also usually means that a lot of boiler plate instrumentation code required to allow tests to set up and validate test scenarios can be omitted from the application. Instead instrumentation code can be injected only where needed for each specific test.

 

Fault injection testing usually relies on transforming the application offline using some sort or code rewriting scheme operating on either the source or binary files. However, with a bytecode based virtual machine like the JVM it is also possible to inject changes into test code dynamically by modifying bytecode either at load time or, possibly even, after it has been loaded. Although this sounds complicated and expensive in fact the JVM's Java agent  capability makes it simple and cheap to do. Also, dynamic transformation has the advantage that application and JVM classes can be  reconfigured appropriate to successive tests by injecting specific fault or instrumentation code before running a  test and then uninjecting it afterwards. By contrast, use of offline source or binary transformation requires either maintaining several different versions of your application built from different versions of the source or else rewriting and then rebuilding the application code or, at least, some subset of it between test runs.

 

Is There a Simple Example of Fault Injection?


Yes, the example shown below is simple and easy to follow. But before showing it to you we need to introduce  the sample application used throughout this tutorial so you can see what code is being broken and what is supposed to happen when you break it.

 

The pipeline processor application is  a small library of classes which allows you to process a stream of input text line by line. Regular expressions can be used to detect text elements such as words or phrases occuring in successive input lines. Matched text elements can be replaced with alternative text and the library also makes it possible to use values obtained from earlier matches to construct the replacement for a later line. Using  a pipeline makes it is easy to perform multiple  match and replace operations as separate stages each of which responsible for a single step in the transformation process.

 

The Pipeline Processor Application

 

At the end of this article you will find details of how to download and install the application source and test code and Byteman rule scripts plus the libraries required to test it with Byteman. The source is organised as a maven project comprising an application source submodule and three test modules. However, the source tree also contains ant build scripts allowing you to compile and package the sources and run the tests using ant. The rest of this tutorial will guide you through the steps required to perform each of these operations.

 

Once you have downloaded and unzipped the application source take a look at subdirectory app. This contains the application library classes and some main program classes which  exercise the library. If you want to build the library you can run command

 

> mvn package

. . .

 

in the top level directory. Alternatively, if you are using ant you can run command

 

> ant build

. . .

 

This should create a directory called target in subdirectory app in which you should find a jar called tutorial2-app-1.0.0.jar

 

To run the example applications change to the app directory and run any of the 3 main classes. For example:

 

> cd app

> java -classpath target/tutorial2-app-1.0.0.jar org.my.PipelineAppMain

 

This first program, PipelineAppMain, uses a file reader as its pipeline source to read text from file foo.txt. This feeds into a multi-stage pipeline which applies 3 distinct substitutions at stages 0, 2 and 4. The pipeline is terminated by a file writer which writes the transformed text to file bar.txt. Two extra  pipeline elements are included at pipeline stages 1 and 3 which direct their input files bar1.txt and bar2.txt. as well as forwarding it to the next stage inline. After running the program you can compare the files to see how each of the substitution stages has itnroduced changes to the text:

 

> diff foo.txt bar1.txt

15c15

<     adinn

---

>     msmith

> diff bar1.txt bar2.txt

6c6

<     Andrew is a member of the Red Hat JVM development team and also

---

>     Michael is a member of the Red Hat JVM development team and also

10c10

<     Andrew has been writing system software for over 25 years, 10 of

---

>     Michael has been writing system software for over 25 years, 10 of

17c17

<     andrew dinn

---

>     Michael dinn

> diff bar2.txt bar.txt

17c17

<     Michael dinn

---

>     Michael Smith

 

As you can see, specific regular expression pattterns are matched  and substituted with replacement text by each pipeline stage.

 

The other main classes you can try out are PipelineAppMain2 and PipelineAppMain3. They use a different matching and substitution model which will be expalined later in the tutorial. These latter  classes uses String inputs and generate String outputs so the main routines print the input, intermediate results and output to System.out. You can browse these classes now to understand what they do or read the rest of this tutorial which will gradually explain their behaviour.

 

The pipeline example was chosen because it provides some interesting opportunities for timing and communication bugs to arise. Each stage of an application pipeline is a Java Thread which reads an input stream (or,  possibly, an external data source) and writes an output stream (or, possibly, an external data sink). The pipeline starts with a data source reader (see interface Source) which can be fed with characters either from a String (see class CharSequenceReader) or from a File (see class FileReader). Successive stages (see class PipelineProcessor) are connected using instances of classes java.io.PipedReader and java.io.PipedWriter. Different subclasses implement different strategies for parsing their input stream and writing transformed text to their output stream. The pipeline ends with a data sink writer (see interface Sink) which collects its output  either as a String (see class CharSequenceWriter) or in a File (see class FileWriter).

 

An application program creates a pipeline one element at a time from source through to sink, connecting them in sequence as it does so. It  executes the pipeline by calling the start() method of each pipeline element (including the source and sink) and then calling their join() method. The source pushes data into the first pipeline stage which transforms it and hands it on to the second stage and so on until it reaches the sink. When there is no more data the source closes its output and exits. Each subsequent stage also exits and closes its output when it detects end of file for its input stream.

 

If an IOException occurs the pipeline processor ignores any remaining input, closes its input and output streams and exits. Note that closing the input is done to ensure that it's upstream pipeline processor exits. If the downstream processor did not close its pipe then the processor feeding it  might end up blocked on a full pipe and hang. Closing the input ensures that the upstream processor receives a broken pipe IOException. Once all the pipeline processors have exited the sink will have collected all the transformed text. You can see an example of how this is set up if you look at the code for class PipelineAppMain.


The library provides several subclasses implementing the base class PipelineProcessor. The latter is an abstract subclass of java.lang.Thread. It implements interfaces Source and Sink, providing methods which allow it to be linked into an upstream Source and to feed a downstream Sink. Its run() method simply delegates to subclasses by calling abstract method processPipeline(). However, PipelineProcessor wraps this call in a try catch finally block to ensure that the input and output pipes are closed where necessary.

 

public abstract class PipelineProcessor extends Thread implements Sink, Source {

    . . .

    public void run() {

        boolean excepted = false;

 

        if (input == null || output == null) {

            throw new RuntimeException("unconnected pipeline");

        }

        try {

            processPipeline();

        } catch (IOException ioe) {

            excepted = true;

        } finally {

            try {

                output.close();

            } catch (IOException ioe) {

            }

            if (excepted) {

                try {

                    input.close();

                } catch (IOException ioe2) {

                }

            }

        }

    }

 

    public abstract void processPipeline() throws IOException;

}

 

PipelineProcessor is subclassed by another abstract class TextLineProcessor. This subclass knows how to split the input text into distinct lines, remembering the end of line convention employed by each line. TextLineProcessor declares an abstract method transform(String) which it calls from its implementation of processPipeline() inside a loop to transform each single line of text (omitting the end of line characters). Subclasses of TextLineProcessor implement this method in order to parse and transform input text lines according to their own needs.

 

public abstract class TextLineProcessor extends PipelineProcessor {

     . . .

    public void processPipeline() throws IOException

    {

        TextLine lineBuffer = new TextLine(input);

        String text = lineBuffer.readText();

        while (text != null) {

            text = transform(text);

            output.write(text);

            if (lineBuffer.isCrLf()) {

                output.write('\r');

                output.write('\n');

            } else if (lineBuffer.isLf()){

                output.write('\n');

            }

            text = lineBuffer.readText();

        }

    }


    public abstract String transform(String line);

    . . .

 

PipelineProcessor also has a subclass called TeeProcessor. This class simply copies its input to two outputs i.e. it allows a sidestream from the main pipeline to be created. This is useful for debugging because the sidestream can be directed to a FileWriter allowing you to see the state of the pipeline stream part way through the transformation process. Program PipelineAppMain uses a couple of TeeProcessors to direct the intermediate output to files bar1.txt and bar2.txt.

 

Breaking a Pipeline Application Using Byteman

 

Before looking at the mechanism used to inject faults into the application let's consider an example pair of Byteman rules which could test class TextLineProcessor  by injecting fault code, in this case exercising the application exception handling:

 

RULE create countdown for TextLineProcessor

CLASS TextLineProcessor

METHOD <init>

IF TRUE

DO createCountDown($0, 2)

ENDRULE

 

RULE throw IOException at 3rd transform

CLASS TextLineProcessor

METHOD processPipeline

AT CALL transform(String)

IF countDown($0)

DO THROW new java.io.IOException()

ENDRULE

 

The first rule is attached to the constructor for TextLineProcessor (<init> is the JVM internal name for a constructor). It will be triggered whenever a TextLineProcessor is constructed. It calls built-in operation createCountDown(Object, int) to create a CountDown instance with counter 2. The first argument, the TextLineProcessor instance $0, is used to label the CountDown, allowing the same CountDown to be retrieved by later rules. As the name suggests this CountDown object is used to ensure that a rule fires when a counting process reaches zero.

 

The second rule is injected into the loop in method processPipeline() just before the call to method transform processes the next line of text. The condition employs the related built-in countDown(Object), passing the TextLineProcessor object $0 to identify the CountDown to use. This built-in only returns true when the CountDown counter has value 0. So, at the first call to transform(String) the counter decrements from 2 to 1 and the condition evaluates to false. This means the rule does not fire and execution returns to method processPipeline() continuing with the call. At the second call the counter decrements from 1 to 0 and the condition is still false so transform(String) is called again. At the third call the counter is 0 so the call to countDown(Object) returns true. The rule fires and throws an IOException, causing the TextLineProcessor to close its input and output streams and exit.

 

So, these two rules simulate an error part way through processing of the pipeline text. Any pipeline stage downstream of this TextLineProcessor should exit after seeing just two lines of text. All upstream pipeline stages which are still active should exit with an IOException.

 

In the next-but-one section we will use the Byteman BMUnit package to inject these two fault rules into a JUnit test. Before doing that we will introduce the BMUnit package using a different example to learn how  to drive the injection process, just for now sticking to rules which simply trace execution.

 

How Do I Inject Trace Into JUnit Tests?

 

The sample application contains a subdirectory called junit, which includes a single  test class BytemanJUnitTests. It contains one test method which exercises a pipeline built using a specific subclass of TextLineProcessor called PatternReplacer. This is the same pipeline processor class as  used by program PipelineAppMain. A PatternReplacer looks for a specific pattern in each given input line and replaces it with some alternative text. Here is how test method testPipeline in class BytemanJUnitTests creates its pipeline


@Test public void testPipeline() throws Exception

{

    System.out.println("testPipeLine:");

    String input = "hello world! goodbye cruel world, goodbye!\n";

    CharSequenceReader reader = new CharSequenceReader(input);

    PatternReplacer replacer = new PatternReplacer("world", "mum", reader);

    CharSequenceWriter writer = new CharSequenceWriter(replacer);

    . . .

 

In this example the pattern is a simple String "world" and the replacement is another String "mum". These values are passed to the PatternReplacer when it is created. The input text is fed into the pipeline by a CharSequenceReader which writes characters from a constant String (actually any implementation of CharSequence will do) downstream to the next pipeline processor in line. This upstream Source also gets passed as an argument when the PatternReplacer  is created. The output text is collected by a CharSequenceWriter which collects incoming data as a sequence of characters. Once the pipeline has completed processing this output text can be retrieved by calling its toString() method.

 

The test method continues as follows

 

    . . .

    reader.start();

    replacer.start();

    writer.start();

    reader.join();

    replacer.join();

    writer.join();

    String output = writer.toString();

    assert(output.equals("hello mum! goodbye cruel mum, goodbye!\n"));

}

 

The test routine must wait for all 3 pipeline stages to complete before it can access the output text. The assertion ensures that the word "world" has been replaced at every occurrence with the word "mum".

 

The Byteman Trace Rules

 

It would be nice to be able to see what is going on when this test gets run so we can see when a line gets transformed. We could insert some TeeProcessor stages into the pipeline but that is problematic on several counts. Firstly, it means we are testing a very different pipeline. Introducing extra pipeline stages will certainly affect the timing of the other pipeline elements and it also means that the stages we are interested in are not directly feeding each other as they would in a real application built using the library.

 

Secondly, adding TeeProcessor stages makes the test code more complicated to follow by introducing elements into the test which are not really necessary for exercising the code we are interested in. This code is only present in order to provide a means of monitoring execution during the test. It would be nice if we could keep this insrumentation code separate from the code driving the test.

 

Thirdly, the TeeProcessor stages can show us what goes into each pipeline stage or what comes out but they cannot show us any intermediate state to explain what is happening as each pipeline element processes its input. In fact, with this application the problem is particularly difficult. Since a pipeline processor is implemented as a self-contained thread the test thread cannot reliably check the state of fields in the pipeline objects at different stages during execution.That would require them to provide some sort of synchronization mechanism so that the test code could stop them executing, idenitfy their state and then allow them to continue.

 

These three issues are not just specific to this example. They are typical of almost every application under test. Luckily, we can use some Byteman to address all three of these problems. We can inject code which observes specific behaviours of the reader, writer and pattern replacer threads in a configuration much closer to the one we really wish to test with no need to clutter our test  code or, indeed application  code, with instrumentation routines.

 

The trace rules we will  use are contained in script trace.btm in directory junit/src/resources:


# A rule which dumps the text passed to a CharSequenceReader when it is created

RULE trace input characters

CLASS CharSequenceReader

METHOD <init>(CharSequence)

AT EXIT

IF TRUE

DO traceln("#INPUT\n" + $1.toString() + "#END")

ENDRULE

 

This first rule dumps the input text supplied when a CharSequenceReader is created. The text is bracketed with #INPUT and #END, allowing us to see what is being fed into the pipeline.

 

# A rule which dumps the text stored in a CharSequenceWriter when it is retreived

RULE trace output characters

CLASS CharSequenceWriter

METHOD toString()

AT EXIT

IF TRUE

DO traceln("#OUTPUT\n" + $! + "#END")

ENDRULE


This second rule dumps the text collected by the CharSequenceWriter bracketed with #OUTPUT and #END when it is retrieved by the test routine, allowing us to see what emerged from the pipeline. Note that the rule is triggered at the end of the call to method toString(). It refers to the value being returned form the call using special variable $!.


# A rule which traces replacements made by a PatternReplacer

# if the returned line of text differs from the input line it

# is dumped to System.out

 

RULE dump pattern transformed text

CLASS PatternReplacer

METHOD transform(String)

AT EXIT

IF NOT $1.equals($!)

DO traceln(" patternReplace(" + $1 + " -> " +$! +")")

ENDRULE


This rule is triggered at the end of the call to method transform(String) of class PatternReplacer. The condition compares the text line provided as argument, $1, with the return value, $!. If these two strings differ then then the rule fires and prints a trace message. If the lines are the same it does nothing.

 

Using BMUnitRunner to Inject the Rules

 

The rules in script trace.btm can be loaded automatically when the test is run by attaching a couple of annotations to the test class:

 

@RunWith(BMUnitRunner.class)

@BMScript(value="trace", dir="target/test-classes")

public class BytemanJUnitTests

{

    . . .

 

The first @RunWith annotation tells JUnit to use the Byteman class BMUnitRunner to run the tests. BMUnitRunner is a subclass of the standard JUnit4 test runner class and it delegates most of the job of running the test to JUnit. However, it operates as a wrapper class loading the Byteman agent when testing begins and loading and unloading Byteman rules as they are needed to run specific tests.

 

In this case the runner detects the presence of an @BMScript annotation on the test class. Before running any of the test methods in the class BMUnitRunner ensures the Byteman agent has been loaded into the current JVM. Next it uploads the rules from the script specified in the annotation, trace.btm, to the agent listener. maven copies this script from the test/resources directory to directory target/test-classes. So, the annotation needs to include the name of the script and the relative path from directory junit. Once all tests in the class have been executed BMUnitRunner connects to the agent listener and unloads all the rules defined in the script.

 

If you are using maven then you can run the tests in module junit from the root directory of your download tree by enabling the junit profile. The profile entry for profile junit in the top-level pom adds subdirectory junit as a module, ensuring that its tests get run as part of the top-level build process

 

> mvn test -P junit

. . .

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running org.my.BytemanJUnitTests

byteman jar is /home/adinn/.m2/repository/org/jboss/byteman/byteman/2.0.0/byteman-2.0.0.jar

testPipeLine:

#INPUT

hello world! goodbye cruel world, goodbye!

#END

patternReplace(hello world! goodbye cruel world, goodbye! -> hello mum! goodbye cruel mum, goodbye!)

#OUTPUT

hello mum! goodbye cruel mum, goodbye!

#END

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.432 sec

 

Results :

 

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

. . .

 

As you can see the Byteman rules are injected and display the input, the replacement operation and the output.

 

If you are using ant then you can run the test from the top level directory with the following command

 

> ant junit

. . .

test:

     [copy] Copying 1 file to /ssd/home/adinn/jboss/byteman/git/tutorial2/junit/target/test-classes

    [junit] testPipeLine:

    [junit] #INPUT

    [junit] hello world! goodbye cruel world, goodbye!

    [junit] #END

    [junit]  patternReplace(hello world! goodbye cruel world, goodbye! -> hello mum! goodbye cruel mum, goodbye!)

    [junit] #OUTPUT

    [junit] hello mum! goodbye cruel mum, goodbye!

    [junit] #END

. . .


Note that for this ant command to work you must ensure that BYTEMAN_HOME is set and identifies the directory where you downloaded and unzipped the Byteman binary release. Also, make sure you have run command ant build first to ensure that the application classes have been compiled and installed in the application jar.


How Do I Configure Injection Into JUnit With Maven?

 

So, how is the maven pom configured to ensure that the Byteman code is available? Essentially you just need to include several byteman jars as test dependencies and set a small number of surefire configuration properties. The file pom.xml in directory junit includes the following dependencies:

 

<dependencies>

    <dependency>

        <groupId>org.my</groupId>

        <artifactId>tutorial2-app</artifactId>

        <version>1.0.0</version>

    </dependency>

    <dependency>

        <groupId>junit</groupId>

        <artifactId>junit</artifactId>

        <version>4.8.2</version>

    </dependency>

    <dependency>

        <groupId>org.jboss.byteman</groupId>

        <artifactId>byteman</artifactId>

        <scope>test</scope>

        <version>${byteman.version}</version>

    </dependency>

    <dependency>

        <groupId>org.jboss.byteman</groupId>

        <artifactId>byteman-submit</artifactId>

        <scope>test</scope>

        <version>${byteman.version}</version>

    </dependency>

    <dependency>

        <groupId>org.jboss.byteman</groupId>

        <artifactId>byteman-install</artifactId>

        <scope>test</scope>

        <version>${byteman.version}</version>

    </dependency>

    <dependency>

        <groupId>org.jboss.byteman</groupId>

        <artifactId>byteman-bmunit</artifactId>

        <scope>test</scope>

        <version>${byteman.version}</version>

    </dependency>

    <dependency>

        <groupId>com.sun</groupId>

        <artifactId>tools</artifactId>

        <version>1.6</version>

        <scope>system</scope>

        <systemPath>${tools.jar}</systemPath>

    </dependency>

</dependencies>

 

The byteman jars required in the path are byteman.jar, byteman-submit.jar, byteman-install.jar and byteman-bmunit.jar. Property byteman.version is defined in the parent pom as 2.0.0. You will need this version or later for the test to work correctly. The test also depends on the app jar and a junit 4 jar. Finally, it includes a dependency on the JVM tools jar which is necessary when running on the Sun or OpenJDK JVMs or on JRockit. This jar provides classes needed in order to dynamically install the Byteman agent.

 

The surefire plugin configuration for the junit pom requires a few tweaks

 

<plugin>

    <groupId>org.apache.maven.plugins</groupId>

    <artifactId>maven-surefire-plugin</artifactId>

    <version>2.4.3</version>

    <configuration>

        <includes>

            <include>org/my/BytemanJUnitTests*.java</include>

        </includes>

        <!-- make sure maven puts the byteman jar in the classpath rather than in a manifest jar -->

        <useManifestOnlyJar>false</useManifestOnlyJar>

        <forkMode>once</forkMode>

        <parallel>false</parallel>

        <!-- ensure we don't inherit a byteman jar from any env settings -->

        <environmentVariables>

            <BYTEMAN_HOME></BYTEMAN_HOME>

        </environmentVariables>

        <systemProperties>

            <property>

                <name>org.jboss.byteman.home</name>

                <value></value>

            </property>

        </systemProperties>

    </configuration>

</plugin>

 

It is important to set useManifestOnlyJar to false in order to ensure that the byteman jar appears in the classpath of the test JVM. This is necessary because loading the agent requires identifying the location of the jar and it is not possible to do this when the jar is made available via a manifest entry.

 

The test should be run in a forked JVM so that the Byteman agent can be loaded into a clean environment. Hence forkMode should be set either to once or always.

 

Tests should be run serially in order to avoid problems where rules for one test introduce side-effects which break code exeucted by an unrelated test. So, parallel shiould be specified with value false.

 

When it loads the agent the BMUnit package will use environment variable BYTEMAN_HOME or System property org.jboss.byteman.home to locate byteman.jar if either of them is set to a non-empty string. Otherwise it scans the classpath to locate the jar. The overrides for these two variables are included in the configuration to ensure that the tests do not accidently pick up the wrong version of byteman.jar. If you are sure these variables are not set you can omit these entries form the configuration.

 

How Do I Configure Injection Into JUnit With Ant?

 

The ant build script build.xml in directory junit shows you how to configure ant to use Byteman. When running with ant you have to ensure the byteman jars are available by downloading and unzipping a Byteman release and setting BYTEMAN_HOME to identify the download directory. The script ensures that BYTEMAN_HOME has been  set and that it identifies a directory containing the expected jar files

 

. . .

<property environment="env"/>

<property name="byteman.home" value="${env.BYTEMAN_HOME}"/>

 

<fail message="please set BYTEMAN_HOME">

    <condition>

        <not>

            <isset property="byteman.home"/>

        </not>

    </condition>

</fail>

 

<fail message="cannot find byteman lib jars">

    <condition>

        <not>

            <and>

                <available file="${byteman.home}/lib/byteman.jar"/>

                <available file="${byteman.home}/lib/byteman-submit.jar"/>

                <available file="${byteman.home}/lib/byteman-install.jar"/>

            </and>

        </not>

    </condition>

</fail>

 

<fail message="cannot find byteman contrib jars">

    <condition>

        <not>

            <available file="${byteman.home}/contrib/bmunit/byteman-bmunit.jar"/>

        </not>

    </condition>

</fail>

. . .

 

The BMUnit jar must be on the classpath when compiling the test code so that references to the byteman annotations and the Byteman test runner class can be resolved.

 

<property name="junit.jar" value="../lib/junit-4.8.2.jar"/>

. . .

<javac srcdir="src/test/java" includes="**/*.java" destdir="target/test-classes" debug="true" target="1.6">

    <classpath>

        <pathelement location="target/test-classes"/>

        <pathelement location="../app/target/tutorial2-app-1.0.0.jar"/>

        <fileset dir="${byteman.home}/contrib/bmunit" includes="byteman-bmunit.jar"/>

        <pathelement location="${junit.jar}"/>

    </classpath>

</javac>

 

Three of the four Byteman jars, byteman-bmunit.jar. byteman-submit.jar and byteman-install.jar,  must be configured on the classpath when you run the JUnit tests. The agent jar, byteman.jar, does not need to be included in the classpath as BMUnit locates and installs it using  environment variable BYTEMAN_HOME.

 

The juit task which runs the tests is cofnigured as follows

 

<junit fork="true" showoutput="true" dir="." forkmode="once">

    <classpath>

        <pathelement location="target/test-classes"/>

        <pathelement location="../app/target/tutorial2-app-1.0.0.jar"/>

        <pathelement location="${junit.jar}"/>

        <fileset dir="${byteman.home}/contrib/bmunit" includes="byteman-bmunit.jar"/>

        <fileset dir="${byteman.home}/lib" includes="byteman-submit.jar byteman-install.jar"/>

        <pathelement location="${java.home}/lib/tools.jar"/>

        <pathelement location="${java.home}/../lib/tools.jar"/>

    </classpath>

    <test name="org.my.BytemanJUnitTests2"/>

    <formatter type="plain"/>

</junit>

 

Notice that the required junit jar is provided as part of the zip download in the top level lib directory.  Also note that the junit tests are forked into a separate JVM. This is needed to allow the Byteman agent to be dynamically loaded.


How Do I Inject Faults Into JUnit Tests?

 

Injecting faults is done in exacttly the same way as you inject trace rules i.e. using annotations provided by the BMUnit package. However, in most cases you want  fault rules to be injected before a specific test and then removed after the test has completed. This is because you normally only want to break the application or Java runtime in a way that is highly specific to the test in question. The junit2 subdirectory contains two further JUnit tests in class BytemanJUnitTests2 which employ rules to break the application. Here is the first method:

 

@RunWith(BMUnitRunner.class)

@BMScript(value="trace", dir="target/test-classes")

public class BytemanJUnitTests2

{

    @Test

    @BMRule(name="throw IOException at 1st transform",

            targetClass = "TextLineProcessor",

            targetMethod = "processPipeline",

            action = "throw new java.io.IOException()")

    public void testErrorInPipeline() throws Exception

    {

        System.out.println("testErrorInPipeline:");

        StringBuffer buffer = new StringBuffer("hello world!");

        buffer.append(" goodbye cruel world, goodbye!\n");

        CharSequenceReader reader = new CharSequenceReader(buffer);

        PatternReplacer replacer = new PatternReplacer("world", "mum",reader);

        CharSequenceWriter writer = new CharSequenceWriter(replacer);

        reader.start();

        replacer.start();

        writer.start();

        reader.join();

        replacer.join();

        writer.join();

        String output = writer.toString();

        assert(output.equals(""));

    }

    . . .

 

If you ignore the different assertion this is essentially the same method as in the previous example. However, this test  employs an @BMRule annotation on the first test method  which injects an exception into the pipeline. By placing the annotation on the method this ensures that the rule is loaded just before running the test and is unloaded just after the test completes. So, this stops the rule affecting the next test method.

 

In contrast to the @BMScript annotation on the class, @BMRule provides the rule text inline in the annotation. The rule is injected into method processPipeline of class TextLineProcessor and it throws an IOException when it is fired. The location field of the annotation is left unspecified so it defaults to  "AT ENTRY". Similarly,  the condition defaults to "TRUE". So, this rule will cause the PatternReplacer to throw an exception as soon as it starts running. The output stream to the writer should be closed with no input having been written. The assert condition checks that the output is indeed an empty String.

 

Notice that the class still has the @BMScript annotation to load the trace rules. These rules are loaded before any tests in the class are run so when this test method and the next test method are executed they will still generate trace showing the input, text replacements and output. You can use either annotation, @BMScript or @BMRule, at the class or method level but this configuration where you have trace rules in a script attached to the class and inline fault rules attached to the test method is the most common one.

 

The second test method is a variant on the method above but it uses a pair of countdown rules like the one we saw earlier.

 

    @Test

    @BMRules(rules={@BMRule(name="create countDown for TextLineProcessor",

                    targetClass = "TextLineProcessor",

                    targetMethod = "<init>",

                    action = "createCountDown($0, 2)"),

                    @BMRule(name="throw IOException at 3rd transform",

                    targetClass = "TextLineProcessor",

                    targetMethod = "processPipeline",

                    targetLocation = "AT CALL transform(String)",

                    condition = "countDown($0)",

                    action = "throw new java.io.IOException()")})

    public void testErrorInStuffedPipeline() throws Exception

    {

        System.out.println("testErrorInStuffedPipeline:");

        StringBuffer buffer = new StringBuffer("hello world!\n");

        buffer.append("goodbye cruel world, goodbye!\n");

        for (int i = 0; i < 40; i++) {

            buffer.append("goodbye! goodbye! goodbye!\n");

        }

        CharSequenceReader reader = new CharSequenceReader(buffer);

        PatternReplacer replacer = new PatternReplacer("world", "mum",reader);

        CharSequenceWriter writer = new CharSequenceWriter(replacer);

        reader.start();

        replacer.start();

        writer.start();

        reader.join();

        replacer.join();

        writer.join();

        String output = writer.toString();

        assert(output.equals("hello mum!\ngoodbye cruel mum, goodbye!\n"));

    }

 

Notice that we have more than one @BMRule annotation here so we use an enclosing @BMRules annotation to bracket them. This is ok when you have two or three simple rules but for larger numbers it is usually clearer to load your rules from a script.

 

These are the same rules as we showed in the earlier example. The first rule creates a countdown when the PatternReplacer is created. The second one throws an exception just before transform(String) is called for the third time. So, in this case the first two lines of text are passed to the CharSequenceWriter and then the output stream is closed. The assertion ensures that only the two expected lines appear in the output.

 

There is another detail here worth considering. The CharSequenceReader posts 40 extra lines into its output. This is enough to fill the piped stream feeding the PatternReplacer, causing the CharSequenceReader to suspend. So, when the exception is thrown the reader should wake up under a suspended write operation and print the resulting pipe full exception to System.err. In the previous test an exception (normally) is not printed because the CharSequenceReader has already written its two lines of input text into the ouptut pipe and exited before the PatternReplacer starts running.

 

Using BMUnitRunner to Inject the Fault Rules

 

The tests in directory junit2 can be run by executing the following command

 

> mvn test -P junit2

. . .

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running org.my.BytemanJUnitTests2

byteman jar is /home/adinn/.m2/repository/org/jboss/byteman/byteman/2.0.0/byteman-2.0.0.jar

testErrorInPipeline:

#INPUT

hello world! goodbye cruel world, goodbye!

#END

#OUTPUT

#END

testErrorInStuffedPipeline:

#INPUT

hello world!

goodbye cruel world, goodbye!

goodbye! goodbye! goodbye!

. . .

goodbye! goodbye! goodbye!

#END

patternReplace(hello world! -> hello mum!)

patternReplace(goodbye cruel world, goodbye! -> goodbye cruel mum, goodbye!)

java.io.IOException: Pipe closed

    at java.io.PipedReader.receive(PipedReader.java:156)

    at java.io.PipedWriter.write(PipedWriter.java:107)

#OUTPUT

hello mum!

goodbye cruel mum, goodbye!

#END

    at org.my.CharSequenceReader.run(CharSequenceReader.java:67)

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.71 sec

Running org.my.BytemanJUnitTests

 

Results :

 

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

. . .

 

If you are using ant you can execute the JUnit fault injection tests from the top level directory wiht the following command

 

> ant junit2

. . .

 

You should see essentially the same output as when you run the maven version of the test.

 

How Do I Inject Synchronization Rules Into TestNG Tests?

 

The BMUnit package can be used with TestNG in almost exactly the same way as it is used with JUnit. The only major difference is in how you inform TestNG that the test requires the use of Byteman rules. We saw that with JUnit you need to add an @RunWith annotation to your test class. For TestMG based tests you need to make your test class inherit from a class called BMNGRunner. This parent class provides all the behaviour needed to ensure that rules are loaded before running tests and unloaded after tests complete.

 

The next example based on TestNG shows some rules which display a timing dependency in the pipeline application. By introducing synchronizations into the threads in the pipeline it is possible to control the order in which specific lines are processed by different pipeline stages. This can be used to prove that given the same pipeline and input the output may vary depending upon the order in which the threads execute. This timing dependency is not normally apparent because the upstream thread in the pipeline tends to process the critical line before the downstream thread starts executing. However by injecting suitable rules we can delay the upstream thread ensuring that the downstream thread runs first or vice versa.


The ability to force threads into uncommon timing windows  is one of the most powerful capabilities provided by fault injection testing. It makes it simple to exercise code reliably and repeatably in circumstances which happen only rarely when the application is deployed. The technique displayed here is applied to a relatively simple and slightly artificial case where there is a clear (and deliberate!) error in the design of the library. However, variations on this technique have proved invaluable for confirming the correctness of, amongst other applications, many  of the multi-threaded libraries which go to make up the JBoss application server.

 

The timing test employs two new subclasses of TextLineProcessor, Binder and BindingReplacer. A Binder passes input lines straight through to its output without modification. However, it  does process the input. The Binder matches elements out of input lines and associates the text with a variable, storing the association, for preference referred to as a binding, in a BindingMap. So, for example, a Binder configured to use pattern "[Aa] ([A-Za-z]+)" and prefix "X" would match the input line "A boy and a dog" twice, binding variable X1 to "boy" and variable X2 to "dog". Note that this example match expression uses a group term (identified by a pair of round brackets) to identify a substring of the matched String to be used as the bound value. If no group term is included then the whole matched value is used.

 

BindingReplacer works in tandem with Binder to insert bound values into text. It uses a BindingMap to lookup values for bound variables and substitutes references to the variables with the relevant values. So, given the input line "The ${X1} chases the ${X2}" and the bindings [X1 -> "boy", X2 -> "dog"] a BindingReplacer would output the transfomed line "The boy chases the dog".

 

The test class BytemanNGTests is located under directory testng. It implements two tests both of which employ the same pipeline and input setup. Here is the one which displays the normal execution order


@BMScripts(scripts = {@BMScript(value="trace", dir="target/test-classes"),

        @BMScript(value="trace2", dir="target/test-classes")})

public class BytemanNGTests extends BMNGRunner

{

    @BMScript(value="timing", dir="target/test-classes")

    @Test

    public void testForwardPublishingGood() throws Exception

    {

        System.out.println("testForwardPublishingGood:");

        BindingMap bindings = new BindingMap();

 

        StringBuffer buffer = new StringBuffer("the boy threw the stick for the dog to catch\n");

        buffer.append("a ${X1} broke a ${X4} with a ${X2}\n");

        buffer.append("the boy threw the stick at the window\n");

 

        CharSequenceReader reader = new CharSequenceReader(buffer);

        Binder binder = new Binder("the ([A-Za-z]+)", "X", bindings, reader);

        BindingReplacer replacer = new BindingReplacer(bindings, binder);

        CharSequenceWriter writer = new CharSequenceWriter(replacer);

        reader.start();

        binder.start();

        replacer.start();

        writer.start();

        . . .

 

The class is annotated with an @BMScripts annotation used to bracket a pair of @BMScript annotations. The first of these annotations loads the trace script trace.btm used in the previous examples.The second one loads an extra trace script trace2.btm which includes rules displaying the operation of the Binder, BindingReplacer and BindingMap.


The test pipeline comprises a Source, a Binder, a BindingReplacer and a Sink. Notice that the binding map created at the start of the method is passed to both the Binder and the BindingReplacer. So, bindings inserted by the Binder can be used by the BindingReplacer.

 

The first input line contains three terms matching the Binder pattern "the ([A-Za-z]+)": "the boy", "the stick" and "the dog". So, when the Binder sees these values it binds the successive variables X1, X2 and X3 to each respective match  updating the binding map to [X1 -> "boy", X2 -> "stick", X3 -> "dog"].  It then passes the line on to the BindingReplacer unchanged. The second line of input does not contain any matching terms so this line is just passed straight through. The third line contains two matches, "the boy" and "the stick", whose matched value is already bound, to variables X1 and X2, respectively. So,  these matches do not result in any new bindings. However, the match for "the window" is new so variable X4 is bound to "window". The bindings are updated to [X1 -> "boy", X2 -> "stick", X3 -> "dog", X4 -> "window"] and line 3 is passed on to the BindingReplacer.

 

The BindingReplacer is not interested in lines 1 and 3 because they do not contain terms of the form ${AANN}. However, it does attempt to transform line 2, substituting values for variables X1, X2 and X4. It should begin to be clear now where the timing problem will occur. Normally, the Binder will process lines 1 2 and 3 before the BindingReplacer is able to process line 2. That means that all four variables will be bound before the BindingReplacer tries to look them up. However, if by some chance the Binder thread is suspended after writing line 2 and the BindingReplacer thread is allowed to run and process line 1 and line 2 then the binding for X4 will not be present in the binding map.

 

So, how can we ensure that a specific order of execution is followed? Well, the critical issue is to control whether or not the Binder processes its third line before the BindingReplacer processes its second line. The first thing we need to be able to do is count which line each of these pipeline elements is processing. Then we need to be able to stop each thread just before it processes the relevant line and then synchronize with it again just after so we can be sure the line has been dealt with. The rules which do this are contained in script timing.btm in subdirectory testng/src/test/resources. Here are the first two rules

 

RULE create Binder rendezvous and counter

CLASS Binder

METHOD <init>

IF TRUE

DO createRendezvous($0, 2, true);

   createCounter($0)

ENDRULE

 

RULE create BindingReplacer rendezvous and counter

CLASS BindingReplacer

METHOD <init>

IF TRUE

DO createRendezvous($0, 2, true);

   createCounter($0)

ENDRULE


Each of these rules calls built-in createRendezvous(Object, int, boolean) to create an object called a Rendezvous (otherwise known as a barrier or thread barrier). The first argument to the createRendezvous call, used to label the Rendezvous so it can be looked up later, is $0, the Binder or BindingReplacer, respectively. So, each of the pipeline elements has its own personal Rendezvous.

 

A Rendezvous is created with a specific count supplied as the second argument, in this case 2, which identifies the number of threads which will meet at the rendezvous. When the Binder or BindingReplacer thread executes one of the rules below which calls the related built-in rendezvous(Object) it may suspend before returning from the call. The rule is that no thread can exit from a call to rendezvous until N threads have entered the Rendezvous where N is the count supplied when the Rendezvous is created. So, in this case when either the Binder or the BindingReplacer thread calls rendezvous it must synchronize with another thread which has also called rendezvous before it can continue. In either case, the other thread is going to be the one running the test. By deciding which order the test thread enters each of these Rendezvous it can control the order in which the Binder and BindingReplacer threads execute. Note that a Rendezvous itself is not ordered -- it doesn't matter whether, say, the test thread or the Binder thread arrives first, just that both threads must be inside a call to rendezvous before either can exit.

 

The third, optional argument to createRendezvous determines whether the rendezvous is deleted at the first exit (the default) or is reset to allow repeated rendezvous operations. For this test we need each thread to enter the rendezvous twice so the argument true is supplied to allow the rendezvous to be re-entered.

 

The second call in the DO list uses built-in createCounter to create a numeric counter. Each counter is labelled using the Binder or BindingReplacer instance, respectively, allowing it to be identified. up in subsequent calls. Counters are normally used to maintain running totals as a program progresses and Byteman provides built-in operations allowing a Counter to be incremented, decremented or read from injected code. In this case the counters are used to track how many input lines have been read.

 

The next rule is used to count lines as they are processed by the Binder and BindingReplacer. This makes it possible to decide when to enter the endezvous


RULE increment Binder or BindingReplacer line counter

CLASS ^TextLineProcessor

METHOD transform

AT ENTRY

IF TRUE

DO incrementCounter($0)

ENDRULE

 

This rule is injected into the transform method declared by TextLineProcessor. However, this is an abstract method. The symbol ^ in front of the class name requests that the rule is injected down the class hierarchy into any implementation defined by a subclass of TextLineProcessor. So, it gets injected into the methods defined by both  Binder and BindingReplacer and whenever a new line of text is presented to either pipeline element its related Counter gets incremented. Notice how $0 is passed as argument to identify the correct Counter instance.

 

The next two rules are used make the Binder enter a rendezvous just before it processes the 3rd line of text and just after it has completed processing the line, respectively

 

RULE enter first Binder rendezvous

CLASS Binder

METHOD transform

AT ENTRY

IF readCounter($0) == 3

DO rendezvous($0)

ENDRULE

 

RULE enter second Binder rendezvous

CLASS Binder

METHOD transform

AT EXIT

IF readCounter($0) == 3

DO rendezvous($0)

ENDRULE


The first rule is injected AT ENTRY so it will be triggered before the line is processed. It only fires when the counter is 3 so the Binder thread will enter the rendezvous for the first time just before it processes the 3rd line. SImilarly, the second rule is injected AT EXIT. So, the Binder thread will enter the rendezvous again just after it has processed the 3rd line.

 

The rules for the BindingReplacer are much the same except that the counter is compared to constant value 2.

 

RULE enter first BindingReplacer rendezvous

CLASS BindingReplacer

METHOD transform

AT ENTRY

IF readCounter($0) == 2

DO rendezvous($0)

ENDRULE

 

RULE enter second BindingReplacer rendezvous

CLASS BindingReplacer

METHOD transform

AT EXIT

IF readCounter($0) == 2

DO rendezvous($0)

ENDRULE

 

The one remaining piece fo the puzzle is  how the test thread is going to enter these rendezvous? After all the test code does not have access to Byteman built-in operations. The answer is that we arrange for the test code to call a dummy method and  inject a call to rendezvous into this dummy method. Here is the rule


RULE allow test thread to enter rendezvous

CLASS BytemanNGTests

METHOD triggerRendezvous

IF TRUE

DO rendezvous($1)

ENDRULE


The rule is attached to method triggerRendezvous of the test class, BytemanNGTests. This method has an empty body so normally calling it would do nothing

 

private void triggerRendezvous(PipelineProcessor processor)

{

    // nothing to do here as Byteman will inject the

    // relevant code into this method

}

 

However, when the Byteman rules are loaded a call to rendezvous will occur, passing $1, the  pipeline element supplied in the call to triggerRendezvous, as argument to the call to rendezvous. The test method can choose to rendezvous with either pipeline processor simply by caling triggerRendezvous.

 

So. here is the rest of the  implementation of test method testForwardPublishingGood

 

        . . .

        triggerRendezvous(binder);

        triggerRendezvous(binder);


        String value = bindings.get("X4");

        assert("window".equals(value));


        triggerRendezvous(replacer);

        triggerRendezvous(replacer);


        reader.join();

        binder.join();

        replacer.join();

        writer.join();


        String output = writer.toString();

        assert(output.equals("the boy threw the stick for the dog to catch\n" +

                            "a boy broke a window with a stick\n" +

                            "the boy threw the stick at the window\n"));

 

    }

 

When the test thread returns from the first call to triggerRendezvous it can be sure  that the Binder is about to start processing the 3rd line. When it returns from the second call to triggerRendezvous it can be sure that the 3rd line has been processed and hence that binding X4 has been added to the bindings map. Hence the assertion which follows these two calls should not fail.

 

The test thread can also be sure that the BindingReplacer has not processed the second line at this point because that cannot happen until the test thread exits from the 3rd call to triggerRendezvous. The BindingReplacer will be forced to wait at this rendezvous until the test thread arrives. The final call to triggerRendezvous allows the BindingReplacer to exit the rendezvous after processing the 3rd line. Since the binding for X4 was present when it executed this line the ouptut should include a substitution for this variable as asserted at the end of the method.

 

The second test method, testForwardPublishingBad, is a minor variation on this first one. It shoulds be obvious how it manages to reverse the execution order.

 

    @BMScript(value="timing", dir="target/test-classes")

    @Test()

    public void testForwardPublishingBad() throws Exception

    {

        System.out.println("testForwardPublishingBad:");

        BindingMap bindings = new BindingMap();

 

        StringBuffer buffer = new StringBuffer("the boy threw the stick for the dog to catch\n");

        buffer.append("a ${X1} broke a ${X4} with a ${X2}\n");

        buffer.append("the boy threw the stick at the window\n");


        CharSequenceReader reader = new CharSequenceReader(buffer);

        Binder binder = new Binder("the ([A-Za-z]+)", "X", bindings, reader);

        BindingReplacer replacer = new BindingReplacer(bindings, binder);

        CharSequenceWriter writer = new CharSequenceWriter(replacer);


        reader.start();

        binder.start();

        replacer.start();

        writer.start();


        triggerRendezvous(replacer);

        triggerRendezvous(replacer);

 

        String value = bindings.get("X4");

        assert(value == null);

       

        triggerRendezvous(binder);

        triggerRendezvous(binder);

 

        reader.join();

        binder.join();

        replacer.join();

        writer.join();


        String output = writer.toString();

        assert(output.equals("the boy threw the stick for the dog to catch\n" +

                            "a boy broke a ${X4} with a stick\n" +

                            "the boy threw the stick at the window\n"));

    }

 

This time the first pair of calls to triggerRendezvous allow the BindingReplacer to transform the second line before the binding for X4 has been inserted. The Binder is not able to process the 3rd line and insert this binding until the 3rd call to triggerRendezvous has been competed. This is reflected in the final output where the reference to variable X4 remains unsubstituted as asserted.

 

To run the tests using maven you execute the following command in the top level directory

 

> mvn test -P testng

. . .

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running TestSuite

byteman jar is /home/adinn/.m2/repository/org/jboss/byteman/byteman/2.0.0/byteman-2.0.0.jar

testForwardPublishingGood:

exits unaccounted for in block B1

#INPUT

the boy threw the stick for the dog to catch

a ${X1} broke a ${X4} with a ${X2}

the boy threw the stick at the window

#END

   bind(X1 -> boy)

   bind(X2 -> stick)

   bind(X3 -> dog)

   bind(X4 -> window)

bindingReplace(a ${X1} broke a ${X4} with a ${X2} -> a boy broke a window with a stick)

#OUTPUT

the boy threw the stick for the dog to catch

a boy broke a window with a stick

the boy threw the stick at the window

#END

testForwardPublishingBad:

#INPUT

the boy threw the stick for the dog to catch

a ${X1} broke a ${X4} with a ${X2}

the boy threw the stick at the window

#END

   bind(X1 -> boy)

   bind(X2 -> stick)

   bind(X3 -> dog)

bindingReplace(a ${X1} broke a ${X4} with a ${X2} -> a boy broke a ${X4} with a stick)

   bind(X4 -> window)

#OUTPUT

the boy threw the stick for the dog to catch

a boy broke a ${X4} with a stick

the boy threw the stick at the window

#END

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.324 sec

 

Results :

 

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

. . .


Alternatively you can execute the following ant command


> ant testng

. . .


How Do I Configure Injection Into TestNG With Maven?

 

The TestNG configuration is very much the same as for JUnit except that a dependency on testng is required.

 

<dependencies>

    <dependency>

        <groupId>org.my</groupId>

        <artifactId>tutorial2-app</artifactId>

        <version>1.0.0</version>

    </dependency>

    <dependency>

        <groupId>junit</groupId>

        <artifactId>junit</artifactId>

        <version>4.8.2</version>

    </dependency>

    <dependency>

        <groupId>org.testng</groupId>

        <artifactId>testng</artifactId>

        <version>5.14.6</version>

        <scope>compile</scope>

    </dependency>

    <dependency>

        <groupId>org.jboss.byteman</groupId>

        <artifactId>byteman</artifactId>

        <scope>test</scope>

        <version>${byteman.version}</version>

    </dependency>

    <dependency>

        <groupId>org.jboss.byteman</groupId>

        <artifactId>byteman-submit</artifactId>

        <scope>test</scope>

        <version>${byteman.version}</version>

    </dependency>

    <dependency>

        <groupId>org.jboss.byteman</groupId>

        <artifactId>byteman-install</artifactId>

        <scope>test</scope>

        <version>${byteman.version}</version>

    </dependency>

    <dependency>

        <groupId>org.jboss.byteman</groupId>

        <artifactId>byteman-bmunit</artifactId>

        <scope>test</scope>

        <version>${byteman.version}</version>

    </dependency>

    <dependency>

        <groupId>com.sun</groupId>

        <artifactId>tools</artifactId>

        <version>1.6</version>

        <scope>system</scope>

        <systemPath>${tools.jar}</systemPath>

    </dependency>

</dependencies>

 

Note that the TestNG version specied here is the earliest which has been found to work.

 

The surefire configuration is essentially the same as for the JUNit tests


<plugin>

    <groupId>org.apache.maven.plugins</groupId>

    <artifactId>maven-surefire-plugin</artifactId>

    <version>2.4.3</version>

    <configuration>

        <includes>

            <include>org/my/BytemanNGTests*.java</include>

        </includes>

        <!-- make sure maven puts the byteman jar in the classpath rather than in a manifest jar -->

        <useManifestOnlyJar>false</useManifestOnlyJar>

        <forkMode>once</forkMode>

        <parallel>false</parallel>

        <!-- ensure we don't inherit a byteman jar from any env settings -->

        <environmentVariables>

            <BYTEMAN_HOME></BYTEMAN_HOME>

        </environmentVariables>

        <systemProperties>

            <property>

                <name>org.jboss.byteman.home</name>

                <value></value>

            </property>

        </systemProperties>

    </configuration>

</plugin>


How Do I Configure Injection Into TestNG With Ant?

 

TestNG provides an ant task allowing you to run TestNG tests which requires much the same configuration optiojns as for runnign a JUnit test. So, the compile and test targets in the TestNG ant build script look much the same as the ones in the JUnit build scripts

 

<property name="junit.jar" value="../lib/junit-4.8.2.jar"/>

<property name="testng.jar" value="../lib/testng-6.3.1.jar"/>

. . .

<taskdef resource="testngtasks" classpath="${testng.jar}"/>

. . .

<target name="compile">

    <javac srcdir="src/test/java" includes="**/*.java" destdir="target/test-classes" debug="true" target="1.6">

        <classpath>

            <pathelement location="target/test-classes"/>

            <pathelement location="../app/target/tutorial2-app-1.0.0.jar"/>

            <fileset dir="${byteman.home}/contrib/bmunit" includes="byteman-bmunit.jar"/>

            <pathelement location="${testng.jar}"/>

        </classpath>

    </javac>

</target>

 

<target name="test" depends="clean, init, compile">

    <copy file="src/test/resources/trace.btm" todir="target/test-classes"/>

    <copy file="src/test/resources/trace2.btm" todir="target/test-classes"/>

    <copy file="src/test/resources/timing.btm" todir="target/test-classes"/>

    <testng workingdir="." outputdir="target/test-output">

        <classpath>

            <pathelement location="target/test-classes"/>

            <pathelement location="../app/target/tutorial2-app-1.0.0.jar"/>

            <pathelement location="${junit.jar}"/>

            <pathelement location="${testng.jar}"/>

            <fileset dir="${byteman.home}/contrib/bmunit" includes="byteman-bmunit.jar"/>

            <fileset dir="${byteman.home}/lib" includes="byteman-submit.jar byteman-install.jar"/>

            <pathelement location="${java.home}/../lib/tools.jar"/>

        </classpath>

        <classfileset dir="target/test-classes" includes="org/my/BytemanNGTests.class"/>

    </testng>

</target>


The taskdef declaration is necessary  to make the testng task available.

 

Notice that the required junit and testng jars are provided as part of the zip download in the top level lib directory. It is not possible to run the TestNG tests form ant using version 5.14.6, the one used in the maven tests. However, it is possible to run tests form ant using version 6.3.1.


Where Can I Download The Tutorial Sources?


The tutorial sources are available from the JBoss downloads server as a zip file.

 

In order to run the example application main programs and  tests provided as part of this downlaod you will need to install a JDK6 release whcih supports the dynamic agent load capability. This includes OpenJDK, Sun's Hotspot JDK and recent releases of the JRockit JDK (version jrmc-4.0.1- is known to work). You should ensure that the Java binaries in the JDK bin directory are available in your PATH.

 

You will also need to install either ant or maven, This tutorial has been tested using ant 7.1 and maven 3.0.2 but it may possibly work with earlier versions. Once again you shoudl ensure the ant or maven binaries are available in your path.

 

 

If you are building and running the test programs with maven you do not need to downlaod and install a Byteman release as maven will automatically pull the relevant jars from the network. However, you will need to configure maven to locate the jars on the public JBoss repository located at

 

http://repository.jboss.org/nexus/content/groups/public

 


If you are building and running the test programs with ant then you will need to download Byteman release 2.0.0 or greater. The latest release is available from the Byteman downloads page

 

http://www.jboss.org/byteman/downloads

 

Unzip the release and set environment variable BYTEMAN_HOME to identify the top-level directory of the resulting directory tree. It should contain several subdirectories including lib and bin.