Bean Validation 1.0 (JSR 303) TCK

 

This page is about the Bean Validation 1.0 (JSR 303) TCK. This testharness is obsolete and succeeded by the Bean Vaildation 1.1 (JSR 349) TCK. If you want information about the latter, you should start reading here. If you are really interested in the JSR 303 TCK keep on reading.

Resources

 

Here are links to the most important pieces for the JSR 303 TCK:

  • The full TCK distribution bundle can be found on SourceForge. The latest version is 1.0.6.GA
<dependency>
     <groupId>org.hibernate.jsr303.tck</groupId>
     <artifactId>jsr303-tck</artifactId>
     <version>1.0.6.GA</version>
 </dependency>
<dependency>
  <groupId>org.hibernate</groupId>
  <artifactId>hibernate-validator</artifactId>
  <version>4.2.0.Final</version>
</dependency>
  • The online documentation for Hibernate Validator 4.2 can be found here.

Running the test harness

 

The actual test for the TCK are defined in the jsr303-tck jar file. You can checkout the source code for the TCK from GitHub as well. The tests are an excellent source for Bean Validation usage examples. Each test is linked to an assertions in the specification via an @SpecAssert annotation. All tested assertions are listed in tck-audit.xml which is part of the TCK.

 

The easiest way to run the TCK is configured as a maven project. We recommend to setup a separate maven module to run the TCK. This module won't contain any source code itself. Instead it will run the tests defined in the Bean Validation TCK against the Bean Validation implementation implementation specified in the POM.

 

You'll need:

  • a pom.xml
    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <parent>
            <artifactId>hibernate-validator-parent</artifactId>
            <groupId>org.hibernate</groupId>
            <version>4.2.0.Final</version>
            <relativePath>../pom.xml</relativePath>
        </parent>
        <artifactId>hibernate-validator-tck-runner</artifactId>
        <name>Hibernate Validator TCK Runner</name>
        <description>Aggregates dependencies and runs the JSR-303 TCK</description>
    
        <dependencies>
            <dependency>
                <groupId>javax.validation</groupId>
                <artifactId>validation-api</artifactId>
            </dependency>
            <dependency>
                <groupId>org.hibernate</groupId>
                <artifactId>hibernate-validator</artifactId>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-api</artifactId>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-simple</artifactId>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.hibernate.javax.persistence</groupId>
                <artifactId>hibernate-jpa-2.0-api</artifactId>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.testng</groupId>
                <artifactId>testng</artifactId>
            </dependency>
            <dependency>
                <groupId>org.hibernate.jsr303.tck</groupId>
                <artifactId>jsr303-tck</artifactId>
            </dependency>
            <dependency>
                <groupId>org.jboss.jbossas.as7-cdi-tck</groupId>
                <artifactId>jbossas-container</artifactId>
            </dependency>
        </dependencies>
    
        <properties>
            <jboss.home>${env.JBOSS_HOME}</jboss.home>
            <validation.provider>org.hibernate.validator.HibernateValidator</validation.provider>
            <remote.debug />
        </properties>
    
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-dependency-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>copy</id>
                            <phase>generate-test-sources</phase>
                            <goals>
                                <goal>copy</goal>
                            </goals>
                            <configuration>
                                <stripVersion>true</stripVersion>
                                <artifactItems>
                                    <artifactItem>
                                        <groupId>org.hibernate.jsr303.tck</groupId>
                                        <artifactId>jsr303-tck</artifactId>
                                        <type>xml</type>
                                        <classifier>suite</classifier>
                                        <overWrite>false</overWrite>
                                    </artifactItem>
                                </artifactItems>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <suiteXmlFiles>
                            <suiteXmlFile>${project.build.directory}/dependency/jsr303-tck-suite.xml</suiteXmlFile>
                        </suiteXmlFiles>
                        <argLine>-Xmx128m</argLine>
                        <forkMode>once</forkMode>
                        <systemProperties>
                            <property>
                                <name>validation.provider</name>
                                <value>${validation.provider}</value>
                            </property>
                            <property>
                                <name>org.jboss.testharness.spi.StandaloneContainers</name>
                                <value>org.hibernate.jsr303.tck.util.StandaloneContainersImpl</value>
                            </property>
                        </systemProperties>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-report-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>generate-test-report</id>
                            <phase>test</phase>
                            <goals>
                                <goal>report-only</goal>
                            </goals>
                        </execution>
                    </executions>
                    <configuration>
                        <outputDirectory>${project.build.directory}/surefire-reports</outputDirectory>
                        <outputName>test-report</outputName>
                    </configuration>
                </plugin>
            </plugins>
        </build>
        <profiles>
            <profile>
                <id>incontainer-debug</id>
                <activation>
                    <property>
                        <name>debug</name>
                    </property>
                </activation>
                <properties>
                    <remote.debug>-Xnoagent -Djava.compiler=NONE -Xdebug
                        -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005
                    </remote.debug>
                </properties>
            </profile>
            <profile>
                <id>incontainer</id>
                <activation>
                    <property>
                        <name>incontainer</name>
                    </property>
                </activation>
                <dependencies>
                    <dependency>
                        <groupId>org.jboss.as</groupId>
                        <artifactId>jboss-as-arquillian-container-managed</artifactId>
                        <scope>test</scope>
                    </dependency>
                </dependencies>
                <build>
                    <plugins>
                        <plugin>
                            <groupId>org.apache.maven.plugins</groupId>
                            <artifactId>maven-surefire-plugin</artifactId>
                            <configuration>
                                <suiteXmlFiles>
                                    <suiteXmlFile>${project.build.directory}/dependency/jsr303-tck-suite.xml</suiteXmlFile>
                                </suiteXmlFiles>
                                <systemProperties>
                                    <property>
                                        <name>java.io.tmpdir</name>
                                        <value>${project.build.outputDirectory}</value>
                                    </property>
                                    <property>
                                        <name>org.jboss.testharness.standalone</name>
                                        <value>false</value>
                                    </property>
                                    <property>
                                        <name>org.jboss.testharness.container.forceRestart</name>
                                        <value>true</value>
                                    </property>
                                    <property>
                                        <name>org.jboss.testharness.runIntegrationTests</name>
                                        <value>true</value>
                                    </property>
                                    <property>
                                        <name>org.jboss.testharness.libraryDirectory</name>
                                        <value>${project.build.directory}/dependency/lib</value>
                                    </property>
                                    <property>
                                        <name>org.jboss.testharness.outputDirectory</name>
                                        <value>${project.build.directory}</value>
                                    </property>
                                    <property>
                                        <name>jboss.options</name>
                                        <value>-Xms128m -Xmx512m -XX:MaxPermSize=512m -enableassertions
                                            -Dvalidation.provider=${validation.provider} ${remote.debug}
                                        </value>
                                    </property>
                                    <property>
                                        <name>org.jboss.testharness.spi.Containers</name>
                                        <value>org.jboss.jbossas.cdiTck.container.ArquillianContainerAdaptor</value>
                                    </property>
                                    <property>
                                        <name>org.jboss.har2arq.container</name>
                                        <value>org.jboss.as.arquillian.container.managed.JBossAsManagedContainer</value>
                                    </property>
                                    <property>
                                        <name>jboss.home</name>
                                        <value>${jboss.home}</value>
                                    </property>
                                    <property>
                                        <name>validation.provider</name>
                                        <value>${validation.provider}</value>
                                    </property>
                                </systemProperties>
                            </configuration>
                        </plugin>
                    </plugins>
                </build>
            </profile>
            <profile>
                <!--
                     Enabling this profile will bundle the BV API and Hibernate Validator w/ the deployed war files instead of
                     testing against the AS provided dependencies
                -->
                <id>bundled-dependencies</id>
                <activation>
                    <property>
                        <name>bundled-dependencies</name>
                    </property>
                </activation>
                <build>
                    <plugins>
                        <plugin>
                            <groupId>org.apache.maven.plugins</groupId>
                            <artifactId>maven-dependency-plugin</artifactId>
                            <executions>
                                <execution>
                                    <id>copy</id>
                                    <phase>generate-test-sources</phase>
                                    <goals>
                                        <goal>copy</goal>
                                    </goals>
                                    <configuration>
                                        <stripVersion>true</stripVersion>
                                        <artifactItems>
                                            <artifactItem>
                                                <groupId>org.hibernate.jsr303.tck</groupId>
                                                <artifactId>jsr303-tck</artifactId>
                                                <type>xml</type>
                                                <classifier>suite</classifier>
                                                <overWrite>false</overWrite>
                                            </artifactItem>
                                            <artifactItem>
                                                <groupId>javax.validation</groupId>
                                                <artifactId>validation-api</artifactId>
                                                <overWrite>true</overWrite>
                                                <outputDirectory>${project.build.directory}/dependency/lib</outputDirectory>
                                            </artifactItem>
                                            <artifactItem>
                                                <groupId>org.hibernate</groupId>
                                                <artifactId>hibernate-validator</artifactId>
                                                <overWrite>true</overWrite>
                                                <outputDirectory>${project.build.directory}/dependency/lib</outputDirectory>
                                            </artifactItem>
                                            <artifactItem>
                                                <groupId>org.slf4j</groupId>
                                                <artifactId>slf4j-simple</artifactId>
                                                <overWrite>true</overWrite>
                                                <outputDirectory>${project.build.directory}/dependency/lib</outputDirectory>
                                            </artifactItem>
                                            <artifactItem>
                                                <groupId>org.slf4j</groupId>
                                                <artifactId>slf4j-api</artifactId>
                                                <overWrite>true</overWrite>
                                                <outputDirectory>${project.build.directory}/dependency/lib</outputDirectory>
                                            </artifactItem>
                                        </artifactItems>
                                    </configuration>
                                </execution>
                            </executions>
                        </plugin>
                    </plugins>
                </build>
            </profile>
            <profile>
                <id>write-artifacts-to-disk</id>
                <activation>
                    <property>
                        <name>dumpArtifacts</name>
                    </property>
                </activation>
                <build>
                    <plugins>
                        <plugin>
                            <groupId>org.codehaus.mojo</groupId>
                            <artifactId>exec-maven-plugin</artifactId>
                            <executions>
                                <execution>
                                    <id>generate-jsr-303-artifacts</id>
                                    <phase>test-compile</phase>
                                    <goals>
                                        <goal>java</goal>
                                    </goals>
                                </execution>
                            </executions>
                            <configuration>
                                <classpathScope>test</classpathScope>
                                <mainClass>org.jboss.testharness.api.TCK</mainClass>
                                <systemProperties>
                                    <systemProperty>
                                        <key>dumpArtifacts</key>
                                        <value>true</value>
                                    </systemProperty>
                                    <systemProperty>
                                        <key>org.jboss.testharness.outputDirectory</key>
                                        <value>target/jsr303-artifacts</value>
                                    </systemProperty>
                                    <systemProperty>
                                        <key>org.jboss.testharness.libraryDirectory</key>
                                        <value>target/jsr303-artifacts/dependencies</value>
                                    </systemProperty>
                                    <systemProperty>
                                        <key>org.jboss.testharness.libraryDirectory</key>
                                        <value>${project.build.directory}/dependency/lib</value>
                                    </systemProperty>
                                    <systemProperty>
                                        <key>org.jboss.testharness.spi.StandaloneContainers</key>
                                        <value>org.hibernate.jsr303.tck.util.StandaloneContainersImpl</value>
                                    </systemProperty>
                                </systemProperties>
                            </configuration>
                        </plugin>
                    </plugins>
                </build>
            </profile>
        </profiles>
    </project>
    
  • and src/main/resources/META-INf/jboss-test-harness.properties
org.jboss.testharness.api.TestLauncher=org.jboss.testharness.impl.runner.servlet.ServletTestLauncher

org.jboss.testharness.testPackage=org.hibernate.jsr303.tck.tests

You can see the full setup for the Hibernate Validator TCK runner here: https://github.com/hibernate/hibernate-validator/blob/4.2.0.Final/hibernate-validator-tck-runner

 

Once you have the right setup you can choose between running the tests standalone or in a container. The example above assumes that you run the TCK against JBoss 7.

Standalone

This is the easiest way to run the test harness.

> mvn clean test

In this case the tests are run with the default surefire classpath. All tests of the TCK which are not marked as @IntegrationTest are run.

 

In container

In order to run against JBoss 7 you need:

  • A JBoss 7 installation
  • Define jboss.home in your POM

If your setup is complete it is enough to run:

 

> mvn clean test -Dincontainer

 

Dumping artifacts

In case you are running against a container the tests are deployed as war/ear files. In order to debug the tests it can be useful to actually inspect the deployed artifacts. To do this just specify the dumpArtifacts property at the command line. This will create all the artifacts which gets deployed in target/jsr303-artifacts.

> mvn clean test -DdumpArtifacts

 

TCK Audit

Part of the source code of the jsr303-tck is an audit tool which allows to determine the coverage of tests in the TCK in relation to the actual specification. The assertions of the TCK are captured in a file called tck-audit.xml which can be found under /src/main/resources of the source code. More information about the format and the usage of this xml file can be found JSR 299 TCK.

 

In order to run the audit tool you have to checkout the jsr303-tck project and run:

> mvn clean test -P tck-audit

 

After the build completes there should be a coverage.html in your target directory. Note, this is no measurements whether an actual implementation passes a TCK. It just indicates how well the TCK tests cover the assertions in the specification. More information about the audit tool can be found here.


What do all the annotations mean?

 

// $Id: BootstrapCustomProviderDefinedInServiceFileTest.java 17345  2009-08-18 09:35:56Z hardy.ferentschik $
/*
* JBoss, Home of  Professional Open Source
* Copyright 2008, Red Hat Middleware LLC,  and individual contributors
* by the @authors tag. See the  copyright.txt in the distribution for a
* full listing of individual  contributors.
*
* Licensed under the Apache License, Version 2.0  (the "License");
* you may not use this file except in compliance  with the License.
* You may obtain a copy of the License at
*  http://www.apache.org/licenses/LICENSE-2.0
* Unless required by  applicable law or agreed to in writing, software
* distributed under  the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES  OR CONDITIONS OF ANY KIND, either express or implied.
* See the  License for the specific language governing permissions and
*  limitations under the License.
*/
package  org.hibernate.jsr303.tck.tests.bootstrap.customprovider;

import  javax.validation.Configuration;
import javax.validation.Validation;
import  javax.validation.ValidatorFactory;

import  org.jboss.test.audit.annotations.SpecAssertion;
import  org.jboss.test.audit.annotations.SpecAssertions;
import  org.jboss.testharness.AbstractTest;
import  org.jboss.testharness.impl.packaging.Artifact;
import  org.jboss.testharness.impl.packaging.ArtifactType;
import  org.jboss.testharness.impl.packaging.Classes;
import  org.jboss.testharness.impl.packaging.IntegrationTest;
import  org.jboss.testharness.impl.packaging.Resource;
import static  org.testng.Assert.assertNotNull;
import static  org.testng.Assert.assertTrue;
import org.testng.annotations.Test;

import  org.hibernate.jsr303.tck.common.TCKValidationProvider;
import  org.hibernate.jsr303.tck.common.TCKValidatorConfiguration;
import  org.hibernate.jsr303.tck.util.TestUtil;

/**
 *  @author Hardy Ferentschik
 */
@Artifact(artifactType =  ArtifactType.JSR303)
@Classes({
        TestUtil.class,
         TestUtil.PathImpl.class,
        TestUtil.NodeImpl.class,
         TCKValidationProvider.class,
         TCKValidatorConfiguration.class,
         TCKValidationProvider.DummyValidatorFactory.class
})
@Resource(source  = "javax.validation.spi.ValidationProvider",
        destination =  "WEB-INF/classes/META-INF/services/javax.validation.spi.ValidationProvider")
@IntegrationTest
public  class BootstrapCustomProviderDefinedInServiceFileTest extends  AbstractTest {

    @Test
    @SpecAssertions({
             @SpecAssertion(section = "4.4", id = "a"),
             @SpecAssertion(section = "4.4.4.2", id = "a")
    })
    public  void testGetFactoryByProviderSpecifiedProgrammatically() {
         TCKValidatorConfiguration configuration = Validation.byProvider(  TCKValidationProvider.class ).configure();
        ValidatorFactory  factory = configuration.buildValidatorFactory();
         assertNotNull( factory );
        assertTrue( factory instanceof  TCKValidationProvider.DummyValidatorFactory );
    }
}

 

@Artifact

Annotation is part of the jboss-test-harness framework and decides which type of artifact should be build when deploying the test into a container. This annotation is irrelevant for when tests are run in standalone mode.

@Classes

Per default only classes from the same package as the test class are included in the artifacts. In order to include other classes the @Classes annotation is used. This annotation is irrelevant for when tests are run in standalone mode.

@Resource(s)

Used to include resource files into artifacts. In case of ArtifactType.JSR303 war files are getting build and the resource files should go somewhere into WEB-INF/classes. This annotation is irrelevant for when tests are run in standalone mode.

@IntegrationTest

Per default tests will be run in standalone as well as in incontainer mode. Marking a test as @IntegrationTest will skip this tests in standalone mode.

@Test

TestNG annotation to mark a method as test.

@SpecAssertion(s)

Used to link a test to the BV specification via the tck-audit.xml file.
Further reading

 

The harness to run the TCK tests is based on and reuses the infrastructure provided by the JSR 299 TCK.