JBossBuild

The JBossBuild build system

 

The jboss repository build system has been deployed for the head and 4.0 branches. This deployment contains the functionality such that the thirdparty folder is created dynamically from components which exist in the repository.

 

Migration Instructions - Head

Check out to a clean location.

 

  • Commit all work or create backups.

 

mkdir test
cd test
cvs co jboss-head
cd jboss-head
cd build
./build.sh

 

Migration Instructions - 4.0 Branch

To pick up the changes you will need to check out the 4.0 branch to a clean location.  A clean location is strongly recommended (if a thirdparty directory already exists it may cause problems).

 

  • Commit all work or create backups.

 

mkdir test
cd test
cvs co -r Branch_4_0 jboss-4.0.x
cd jboss-4.0
cd build
./build.sh

 

Migration Instructions - 3.2 Branch

To pick up the changes you will need to check out the 3.2 branch to a clean location.  A clean location is REQUIRED!  Perform a full checkout to an empty directory.

 

  • Commit all work or create backups.

 

mkdir test
cd test
cvs co -r Branch_3_2 jboss-3.2.x
cd jboss-3.2
cd build
./build.sh

 

 

This will download the thirdparty dependencies and then do an initial build.  The migration process is now complete.

 

 

To suppress the download of thirdparty items you may issue the command:

 

./build.sh -Dinhibit.downloads=true

 

-


 

FAQ

 

1)  I am seeing this error:

 

/jboss-4.0/thirdparty/sun-jmf/lib/jmf.properties;

java.util.zip.ZipException: error in opening zip file

 

What is wrong?

 

Answer:  You are getting a collision between the "old" thirdparty folder and the "new" dynamically built one.   Check out into a new clean workspace.

 

mkdir test
cd test
cvs co -r Branch_4_0 jboss-4.0.x
cd jboss-4.0
cd build
./build.sh

 

 

2)  I have a problem compiling ....

 

Did you check out using the module name jboss-4.0 ?  This is incorrect.

Check out the module jboss-4.0.x    This is the correct module to use.

 

3) How can i test a local binary without it being overwritten?

 

The JBoss build system uses the ANT get task with the usetimestamp attribute to retrieve remote files.   This will download an HTTP file ONLY if it is newer than the local copy.  As a safety mechanism, we have built in a means to instruct the system to NOT download any dependencies.  To test a local binary in this manner your procedure might be as follows:

 

  1. Update your source tree and build, letting the build system retrieve the latest versions of dependencies.

  2. Copy your new binary library to the appropriate thirdparty folder.

  3. Rebuild the source tree using the -Dinhibit.downloads=true parameter.

    1. cd /build

    2. ./build.sh -Dinhibit.downloads=true

  4. The build will be executed without attempting to retrieve any binaries.

  5. After testing is the binary will be moved to the appropriate folder in the repository.jboss.org module and committed.

 

4) Which CVS module should I use to checkout JBoss?

 

See JBossASCVSModules

 

5) How can I test a new component entry in the repository?

 

  1. Checkout from svn the folder repository.jboss.org:

    svn co https://svn.jboss.org/repos/repository.jboss.org

     

  2. Modify the local copy of the repository in the way you wish

  3. Run a build, overriding the property jbossbuild.repository to point to your local copy of the repository:

    ./build.sh -Djbossbuild.repository=file:///home/username/repository.jboss.org

     

  4. Review the contents of thirdparty to see if they are what you expect

 

6) Should I use Java 5 or Java 6 when building?

 

You can use either Java 5 or Java 6, but the primary target is Java 5, so it's always worth checking your work (run the build, execute the smoke tests) on 5

 

-


 

Repository File Structure

repository.jboss.org

 

repository.jboss.org\licenses

 

repository.jboss.org\component name

 

repository.jboss.org\component name\version

 

repository.jboss.org\component name\version\lib

 

repository.jboss.org\component name\version\resources

 

repository.jboss.org\component name\version\resources\dtd

 

repository.jboss.org\component name\version\resources\schema

 

repository.jboss.org\component name\version\bin

 

repository.jboss.org\component name\version\api

 

 

licenses folder- contains all of the licenses in the format name.txt

 

Component name folder - contains all of the version folders, nothing else

 

Version folder - contains component-info.xml, readme's, does not contain licenses

 

 

lib folder - contains files with extensions jar, zip, sar, rar, war, deployer

 

resources folder - contains files with extensions xml, policy, properties, aop, idl, ddl

 

resources/schema folder - contains files with extension xsd

 

resources/dtd folder - contains files with extension dtd

 

bin folder - contains files with extensions sh, bat, conf, md5

 

api folder - contains files with extension api

 

 

Note: only files in the lib directory and resources directory have pathelements defined in libraries.ent (see below).

 

 

 

 

Thirdparty Dependency and Maintenance

 

The new build pulls in binaries from the thirdparty repository. The contents of this http server are populated from the repository.jboss.org svn module every 10 minutes.

 

Instructions for updating repository

 

  1. Checkout the module repository.jboss.org from svn

    1. svn co https://svn.jboss.org/repos/repository.jboss.org repository.jboss.org

  2. Find the folder you wish to add to, or create a new one

    1. For example,

      thirdparty/jboss/aop/lib/*

      would map to

      repository.jboss.org/jboss/aop/1.1.3/lib/*

      for the 1.1.3 aop release.

    2. There may be some confusion over which folder to add binaries to (e.g juddi vs. juddi-juddi).  The 4.0 build used components without the repeated name.   The folders with the double naming schema will be removed.

  3. Create a new version number folder.

    1. Generally, one should never update existing binaries. The only time this should be done is for snapshots on pre-release binaries.

    2. If you are including a new version number (which you should be) please use the following schema for consistency:

      1. Dependency versions should follow this format X.YY.ZZ (i.e. 1.2.3)

      2. X signifies major version related to production release. YY signifies minor version with minor feature changes or additions. ZZ signifies patches and bug fixes.

  4. Create a lib folder to house your jar.  At this point your structure should be component name/version/lib

  5. Add your jar(s) to the lib folder.

  6. Add a component-info.xml file to the version directory.

    1. This includes the version, home page, license type, artifacts, imports, and exports.  See the sample component-info.xml

    2. Please ensure you are providing a projectHome, licenseType, and description.

  7. Include or update your component in /build/build-thirdparty.xml e.g. /build/build-thirdparty.xml

    <componentref name="jboss/aop" version="1.1.3" ></componentref>

     

  8. Commit the new build-thirdparty.xml file

 

 

Note:  Because of server synchronization it may take up to 10 minutes for your change to propogate through the system. For a full example of a build-thirdparty.xml and its output see ExampleBuildThirdparty.

 

The next time you execute build.sh the new dependency will be downloaded.

 

Sample component-info.xml:

<project name="jboss-aop-component-info">

   <!-- The component id is the repository path -->
   <component id="jboss/aop"
      <!-- The license type for which there must be an *.txt in repository.jboss.org/licenses -->
      licenseType="lgpl"
      <!-- The version of the component -->
      version="1.1.3"
      <!-- The component home/info page -->
      projectHome="http://www.jboss.org/products/aop"
      <!-- A short description of the component -->
      description="JBoss Aspect Oriented Programming">

      <!-- Artifacts produced by this component -->
      <!-- Each listed artifact is copied from repository.jboss.org to thirdparty -->
      <!-- Directory location determined by filename extension -->
      <artifact id="jboss-aop.jar"></artifact>
      <artifact id="jboss-aop-jdk50.jar"></artifact>
      <artifact id="pluggable-instrumentor.jar"></artifact>
      <!-- Components we depend on -->
      <import componentref="javassist">
         <compatible version="3.0beta"></compatible>
      </import>
      <!-- The artifacts exported by this component (lib entries only) -->
      <!-- Entries are added to the build classpath via libraries.ent -->  
      <export>
         <include input="jboss-aop.jar"></include>
         <include input="jboss-aop-jdk50.jar"></include>
         <include input="pluggable-instrumentor.jar"></include>
      </export>
   </component>

</project>

 

-


 

Instructions for migrating an external projects build to use the repository download

 

  1. You will need two files from jboss-head

    1. /tools/etc/jbossbuild/tasks.xml

    2. /tool/lib/jbossbuild.jar

  2. Place these files in an identical location in your own source tree

  3. Edit your main build.xml and add two targets such as the following:

    1. target: createthirdparty will be the first target you should execute in your build

    2. it will download the required thirdparty items and create a file in /thirdparty called libraries.ent

    3. the libraries.ent file will contain property definitions for the thirdparty items

 

 <!-- create the thirdparty folder from items in the repository -->
   <!-- then generate a new libraries.ent file and include it in  -->
   <!-- the build                                                 -->
   <target name="createthirdparty" unless="inhibit.downloads"
      depends="check.inhibit.downloads">
      <ant antfile="build-thirdparty.xml" target="generate-lib-file"></ant> 
   </target>    
 
   <!-- check if thirdparty libraries are to be downloaded -->  
   <target name="check.inhibit.downloads">
      <condition property="inhibit.downloads">
         <or>
            <uptodate property="dependencies.current"
            srcfile="build-thirdparty.xml"
               targetfile="../thirdparty/libraries.ent"></uptodate>
            <istrue value="${nodownload}"></istrue>
         </or>
      </condition>
  </target>       

 

  1. Next, you need to create a build-thirdparty.xml file.   See the existing one in jboss-head/build for an example.  Simply replace the <componentref> definitions with the ones required by your project.

  2. The only caveat of this process is that since the libraries.ent file is generated at runtime you cannot define properties which will be contained in it in the main build file.

 

Generating JDK1.4 Compatible Jars

See the JBossRetro project.

 

Building the jbossbuild.jar and how to perform maintenance

 

The jbossbuild project lives in a cvs module called jbossbuild.   The development currently takes place in a branch with the name BRANCH_JBBUILD_DEV. Tagged snapshots are made on this branch which correspond to jbossbuild releases. Most likely, you will either wish to check out the last released snapshot or the head of the development branch.

 

The current version of jbossbuild.jar used in jbossas trunk is JBossBuild-1.1.5 (from tag JBossBuild_1_1_5)

 

To determine the last released snapshot, look at the jira for the jbossbuild project and find the last released version.  This is available under the changelog.

 

JBossBuild Project Changelog

 

For example, to check out the version tagged as JBossBuild_1_1_5 you would issue the command:

 

cvs co -r JBossBuild_1_1_5 jbossbuild

 

Once you have the last release checked out (into eclipse for example) you can begin making any necessary changes.

 

Here are a list of classes which are often your starting point:

  1. SynchronizeComponentsTask.java - this is your starting point.  This is the ant task which is called in build/build-thirdparty.xml.   This class loops through all of the dependencies and retrieves their component-info.xml files.  This class kicks of the graph visiting mechanism.

  2. ComponentRefGraphFileResolverVisitor - compiles a list of files to resolve

  3. ComponentRefGraphChildResolverVisitor - resolves transient dependencies

 

 

So, depending on what the change you need to make is, modify the relevant classes.

 

  1. Test your changes

    1. cvs co jboss-head

    2. Create a jbossbuild.jar

      1. In eclipse in your jbossbuild project click on src/main which highlights it

      2. File->export->Jar file then click the next button

      3. select your project, then click the export classes and resource files box

      4. select your output path then click finish

    3. Copy the newly created jbossbuild.jar to your jboss distribution

    4. delete the thirdparty directory if it exists, then build jboss making sure your changes do not cause errors

  2. Commit your changes

    1. Use eclipse to commit your changes to the src files

      1. First you will need to do an update against the dev branch (you can't commit on a tag)

      2. Right click on the src file, click team -> update

      3. Select BRANCH_JBBUILD_DEV

      4. Now commit your change after the update has taken place

      5. Right click on the src file, click team -> commit

      6. Do this for all modified source

    2. Now you need to tag this release of jbossbuild

      1. Right Click on the root of the jbossbuild project

      2. Select team -> tag as version

      3. Enter a tag name

      4. Do any necessary jira work (marking this release as complete)

    3. Commit your changes in the jboss distro

      1. Make sure you keep branch_4_0 and head in sync with same version of jbossbuild

    4. Add this version of jbossbuild to the repository.jboss.org (see repository instruction on this wiki)

    5. Location is jboss/jbossbuild