Embedded Jopr QA

Internal QA info: http://intranet.corp.redhat.com/ic/intranet/TestingEmbeddedConsoleWithAS.html

Testopia plan: https://testopia.devel.redhat.com/testopia/tr_show_plan.cgi?plan_id=676

QA JIRA issue: https://jira.jboss.org/jira/browse/JBQA-1938

Tracking issue: https://jira.jboss.org/jira/browse/JBQA-2498

EAP tracking issue: https://jira.jboss.org/jira/browse/JBPAPP-2422

EAP 5 test matrix: https://jira.jboss.org/jira/browse/JBQA-2549

 

Embedded Jopr

JIRA repository: https://jira.jboss.org/jira/browse/EMBJOPR

Hudson jobs list: http://hudson.qa.jboss.com/hudson/view/EmbJopr/

Hudson job: http://hudson.qa.jboss.com/hudson/job/embjopr/

SVN: http://svn.jboss.org/repos/embjopr/trunk/

Changelog: http://fisheye.jboss.org/changelog/EMBJOPR/

Releases:

  r784 = EmbJopr 1.3.2.GA

  r683 = EmbJopr 1.3.1.GA

Jopr

JIRA repository: https://jira.jboss.org/jira/browse/JOPR

Hudson job: http://hudson.qa.jboss.com/hudson/job/rhq-jopr-nightly/

SVN: http://anonsvn.jboss.org/repos/jopr/

Changelog: http://fisheye.jboss.org/changelog/JOPR/

 

Branches

http://fisheye.jboss.org/changelog/EMBJOPR/branches

Hudson jobs - AS-5.1.x:

default JDK5: http://hudson.qa.jboss.com/hudson/job/JBoss-AS-5.1.x-EMBJOPR-JSFUNIT/ (last, TS.txt, workspace/target)

standard JDK5: http://hudson.qa.jboss.com/hudson/job/JBoss-AS-5.1.x-EMBJOPR-JSFUNIT-standard/ (last)

all JDK5: http://hudson.qa.jboss.com/hudson/job/JBoss-AS-5.1.x-EMBJOPR-JSFUNIT-all/ (last)

default JDK6: http://hudson.qa.jboss.com/hudson/job/JBoss-AS-5.1.x-EMBJOPR-JSFUNIT-default-JDK6/ (last)

default JDK5 IE7: http://hudson.qa.jboss.com/hudson/job/JBoss-AS-5.1.x-EMBJOPR-JSFUNIT-ie7/

Hudson jobs - EAP-5.0.x:

default JDK5: http://hudson.qa.jboss.com/hudson/job/JBoss-EAP-5.0.x-EMBJOPR-JSFUNIT/ (last, TS.txt, workspace/target)

standard JDK5: http://hudson.qa.jboss.com/hudson/job/JBoss-EAP-5.0.x-EMBJOPR-JSFUNIT-standard/ (last)

all JDK5: http://hudson.qa.jboss.com/hudson/job/JBoss-EAP-5.0.x-EMBJOPR-JSFUNIT-all/ (last)

default JDK6: http://hudson.qa.jboss.com/hudson/job/JBoss-EAP-5.0.x-EMBJOPR-JSFUNIT-default-JDK6/ (last)

default JDK5 IE7: http://hudson.qa.jboss.com/hudson/job/JBoss-EAP-5.0.x-EMBJOPR-JSFUNIT-ie7/

 

 

Current versions

We're testing with (versions [and reasons for that version to be chosen]):

 

JBoss AS 4:   jboss-eap-4.3.0_CP03

EmbJopr:       1.1.0.GA

 

JBoss AS 6:   jboss-6.0.0.Mx

EmbJopr:       1.2.0.GA

 

JBoss EAP 5:    EAP 5.0 branch  - https://svn.jboss.org/repos/jbossas/branches/JBPAPP_5_0/

EmbJopr:         1.3.2

 

JDKs:              SUN 1.6.0 - latest update (currently 17)

                       OpenJDK 1.6

                       IBM JDK 1.6

 

JSFUnit:          1.0.0.GA

 

Versions

 

EmbJopr

(jopr-embedded-parent)

RHQ parent
EAP / AS
JSFUnit
HtmlUnit
1.3.2.GAEAP 5.01.0.0.GA2.3
1.3.3.SNAPSHOTAS 6.0 trunk1.1.0.GA

2.6

1.3.3.SNAPSHOT with RF 3.3.3 SNAS 6.0 M1--
JSFUnit d/n support HTMLUnit 2.61.3.3.SNAPSHOTAS 6.0 M11.1.0.GA2.6
FacesException: Can't instantiate HtmlAjaxRepeat1.3.3.SN r 861 w/ RF 3.3.3.BETA11.4.0-SNAPSHOTEAP 5 GA
EMBJOPR-271 - CNFEx: ManualAddFacet1.4.0-SN r8611.4.0-SN r5264EAP 5 GA

EMBJOPR-273 - Compile errors:

ContentSourcePackageDetailsKey

1.4.0-SN (2010-01-14)1.4.0-SN (2010-01-14)
Finally the TS is able to run.1.4.0.SN r8661.4.0-SN fd9664ea0c7a8053f367e1b4d1bc69ddca0a9be0EAP 5 GA1.1.0.GA-HU262.6
Finally the TS is able to run.1.4.0.SN r8661.4.0-SN a9cd9f570564802a56eea486396ace806feb2d44EAP 5 GA1.1.0.GA-HU26

2.6

After disabling navtree ajax, it runs once again.From EAP 5 GA zip

EAP 5 GA zip1.2.0-Final2.7

 

Docs

HTMLUnit:

     http://htmlunit.sourceforge.net/faq.html#AJAXDoesNotWork

     http://htmlunit.sourceforge.net/changes-report.html

Mozilla Rhino:

     http://www.mozilla.org/rhino/

 

JBoss AS 5 Test Execution Summary

2009-May-17

Tests run: 222, Failures: 38, Errors: 0, Skipped: 0

 

Of the 38 failures:

 

  2 testAppsPagination/testAppsListing - EMBJOPR-61
  1 testWarHotDeployManyWarsAtOnce

  5 Connection Factory Tests JOPR-191/JBAS-6918

23 Datasource/ JMS Metrics/Operations Tests: JOPR-141/JBAS-6720

 

  4 Start/Stop/Restart EAR/WAR/RAR failing - EMBJOPR-172 - Test logic needs adjustment - Fix in progress

  1 testRarConfigurationTab - To Be Excluded, same as EAR/WAR below

  2 ServerNodeSummaryTest: EMBJOPR-174 - To be Excluded - Not Implemented

 

 

Tests Currently Excluded as Features will Not Be Implemented for GA

4 WAR/EARConfigurationTab: EMBJOPR-96/JOPR-147

 

 

How to run Embedded Jopr tests

  1. Download and unzip JBoss AS / EAP distribution.
  2. Set the JBOSS_HOME property in maven's setting.xml to that JBoss directory.
  3. Checkout embjopr project from SVN.
  4. Set the root project and jbas4 + jbas5 modules to skip tests in the IDE of choice, or use -Dmaven.skip.test=true in later steps 3 and 4. This is because Seam tests fail
  5. Build the root project recursively (this makes Maven downloads dependencies). This also builds the jbas4 / jbas5 module.
  6. mvn clean install
  7. Build the jsfunit module with the jboss4.2 or jboss5x profile
  8. cd jsfunit
    mvn clean install -Pjboss5x
    cd ..
    

 

The last step includes launching JBoss AS in the JBOSS_HOME directory using the Maven Cargo plugin. Do not deploy the resulting war manually on that server, otherwise Cargo will behave quite weirdly (error parsing ServletRedirector.jsfunit, 404 Not found etc). If getting exceptions like this, make sure there's not the war deployed accidentally (in my case, it happened when I invoked non-standard maven goal; thanks to Farah Juma for pointing this out).

 

How to run tests against manually started EAP / AS

 

  1. Build the WAR using the jsfunit project module.
  2. Copy <project>/jsfunit/target/admin-console-jsfunit.war directory to EAP's deploy dir.
  3. Start EAP.
  4. Optional: Verify that the admin console works; restart EAP.
  5. Run the testsuite without -Dcargo - that will skip starting of EAP and deployment of the built WAR.

 

 

Script to download and build prepare the projects

##  Download and build EmbJopr
mkdir embjopr-svn-trunk
cd embjopr-svn-trunk
svn co http://anonsvn.jboss.org/repos/embjopr/trunk/ .
mvn clean install -DskipTests
cd ..

##  Download and build JBoss AS / EAP
mkdir AS6-trunk
ln -s AS6-trunk JBOSS_CODE
cd JBOSS_CODE
svn co http://anonsvn.jboss.org/repos/jbossas/trunk .
cd build
./build.sh clean
./build.sh -Dbuild.unsecure=true
cd ../..

##  Soft-link the binaries
JBOSS_BUILD=`pwd`/`ls -d -1 jbosseap-5.x/build/output/* | tail -n 1`
ln -s $JBOSS_BUILD JBOSS_BUILD
ln -s JBOSS_BUILD JBOSS_HOME
export JBOSS_HOME=`pwd`/JBOSS_HOME
echo "JBOSS_HOME for settings.xml: $JBOSS_HOME"


##  Remove bundled EmbJopr
rm -rf JBOSS_HOME/server/default/deploy/admin-console.war
rm -rf JBOSS_HOME/server/all/deploy/admin-console.war
rm -rf JBOSS_HOME/server/standard/deploy/admin-console.war
rm -rf JBOSS_HOME/server/production/deploy/admin-console.war

 

See also: JSFUnitWithMaven

 

Running tests on Hudson: (TBD)

 

Example settings.xml

 

<?xml version="1.0" encoding="UTF-8"?>
<settings>
  <profiles>
    <profile>
      <id>jboss4.2</id>
      <properties>
        <JBOSS_HOME>/home/ondra/work/JOPRembedded/jboss-eap-4.3/jboss-as</JBOSS_HOME>
        <JAVA_HOME>/home/ondra/sw/java/jdk/jdk1.5.0_17</JAVA_HOME>
      </properties>
    </profile>     
    <profile>
      <id>jboss5x</id>
      <properties>
        <JBOSS_HOME>/home/ondra/work/JOPRembedded/jboss-5.0.0.GA</JBOSS_HOME>
        <JAVA_HOME>/home/ondra/sw/java/jdk/jdk1.5.0_17</JAVA_HOME>
      </properties>
    </profile>
  </profiles>
</settings>

 

 

How to run a single / selected test class

mvn -Pjboss5x -Dtest=org.jboss.jopr.jsfunit.as5.DatasourceTest

 

In fact, test property takes a mask as it's value, so you can use asterisk to filter a group of tests:

 

mvn -Pjboss5x -Dtest=Datasource*Test

 

See http://maven.apache.org/plugins/maven-surefire-plugin/examples/single-test.html.

 

If you need to run single method, there's no (known) way to do it.

Running tests against a specific JBoss configuration

mvn -Pjboss5x -Dcargo.jboss.configuration=${configName}

 

replacing configName with all or standard

 

The test run defaults to running against default config.

 

JBoss AS / EAP manual launch options

 

To expose JMX (shows JVM metrics):

 

-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9002 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.awt.headless=true

 

To run SUN JVM debugger:

-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=4000,server=y,suspend=y

 

AS 5 issues when running JSFUnit tests with Maven Cargo plugin

 

Tests architecture

The tests are designed to cover both AS 4 (written using EAP 4.3.0) and AS 5 using the same code base, and covering the differences by overriding base classes.

Package hierarchy
  • org.jboss.jopr.jsfunit
    • org.jboss.jopr.jsfunit.as4
    • org.jboss.jopr.jsfunit.as5
      • org.jboss.jopr.jsfunit.as5.app
        • org.jboss.jopr.jsfunit.as5.app.ear
        • org.jboss.jopr.jsfunit.as5.app.war
        • org.jboss.jopr.jsfunit.as5.app.rar
        • org.jboss.jopr.jsfunit.as5.app.ejb
      • org.jboss.jopr.jsfunit.as5.datasources
      • org.jboss.jopr.jsfunit.as5.jmsDestinations

 

The org.jboss.jopr.jsfunit package contains the base class EmbjoprTestCase with supporting methods which deal with common tasks of EmbJopr manipulation.

 

Other classes in this package, like DatasourceTestBase, inherit from the EmbjoprTestCase class. Generally, they follow the naming pattern <EmbJoprCompomentName>TestBase. These sub-classes contain test methods "stubs". They should provide at least general test abstraction, but usually they will also perform most of the test, as far as this code would be the same for AS 4 and AS 5.

 

The differences between tests for AS 4 and AS 5 are coded in subclasses of the *TestBase classes in the as4 and as5 packages. In case there's no difference between behavior of AS 4 and AS 5, the sub-class is not needed to override corresponding method; but, for clarity, it's better to override and explicitly call super.testSomething() method.

 

 

AS 4 and AS 5 test co-design procedure

  • One of us, let's call him Lumberer, will put new test to as4 or as5 package (assuming that it builds and passes).
  • Other of us, let's call him Carpenter, will review it, try it to run against the other AS version and manualy check the differences between EmbJopr for AS 4/5.              
    • If there's no difference, the test method can be copied to the *TestBase class.               
      • In the as#.*Test class, it can be removed, or replaced with trivial override:
        @Override protected void testSomething() { super.testSomething(); }
    • If there is a small difference, Carpenter will create an _abstract_ helper method, and move the part of the code which he needs to change to Lumberer's corresponding as# package. Then, carpenter will also override it in his/her as# package.               
      • Lumberer can be aware of the difference in advance, so he can eventually create abtract helper method and override it in the Carpenter's package with a method just throwing UnsupportedOperationException.   
      • Carpenter can also use simple if( isEap4() ){ ... } else { ... }, but the drawback would be that we will have tests for two AS versions. What if future AS versions will be different again? But for trivial cases like client.click( isEap4() ? "Continue" : "Save" ) I thing this is ok and saves us some complexity.   
       
    • If there's a big difference that is probably not easily splittable, Carpenter will create an abstract testMethod(), leave whole test code in Lumberer's package, and create his/her own implementation.   
    • The same applies to various helper methods serving more tests. For the methods in EmbjoprTestCase, currently there's no subclassing, but I suppose those methods will be generic enough to work with both AS versions. If not, we will solve it later.

 

Test coverage & plan

See JBoss Jira JBQA-1938.

 

WAR

Moved to a Embedded Jopr QA - WAR Tests.

 

Enterprise Application (EAR)

Moved to a Embedded Jopr QA - WAR Tests.

 

EJB

Moved to a Embedded Jopr QA - WAR Tests.

 

 

See also: