Version 8
Created by ozizka on Nov 15, 2011 11:15 PM. Last modified by dlofthouse on Feb 21, 2012 6:49 AM.

    This should be a concise list of requirements for the AS 7 testsuite.

    There are some documents already, plus discussions on mailing list

    This should be a shorter aggregation of all of them, kind of check-list.

    It's possible this will then be moved elsewhere (AS docs?).

     

    Common requirements:

     

    1. The default build (i.e. mvn install) must run the smoke tests
    2. There must be a way to run all tests
    3. There must be a way to run a group of tests
    4. There must be a way to run individual tests
    5. Client side test debuging must work
    6. It should be possible to run individual tests in the IDE
    7. It should be possible to run indivudual tests againt an already running server
    8. It must be possible to skip the test run

     

    Common solutions

     

    1. mvn install
    2. mvn -DallTests install
    3. mvn -DskipTests  to skip all tests incl. unit tests of all modules and whole testsuite.
    4. ./build.sh clean install
    5. ./build.sh install
    6. ./build.sh install -DskipTests
    7. ./build.sh clean install -DskipTests
    8. ./build.sh clean install -DallTests

     

    QA dept's testsuite requirements: https://docspace.corp.redhat.com/docs/DOC-69049 , JBQA-4729

     

    With these priorities:

    •   AS7-2541  TS: Clutering tests - figure out how they will be run.
    •   AS7-2485  Propagate property node0 to system properties
    •   AS7-2086  TS: Tests grouping
    • AS7-2540  TS: Move *-arquillian.xml from testsuite.integration/src/test/resources/** to some better place.
    •   AS7-2210  TS: Move Arquillian configs from src/test/resources/X to some special dir.
    •   AS7-2448  TS: Test output is not stored in ...-output.txt
    •   AS7-2176  TS: Check tests workdirs
    •   AS7-2166  Alternative JDKs for building and running - OpenJDK 6, 7, Sun JDK 7, IcedTea

     

    Feel free to add additional requirements.

     

    Tests should also fail if their initialisation or clean up risks leaving the server in an inconsistent state - as an example we currently have tests that will silently ignore errors adding or removing security realms despite the fact that using the correct realm is critical to the correct execution of the test.

    Would also suggest a unique naming of all resources used so if one test leaves a server in an inconsistent state it will not conflict with a different test.