I will highly appreciate any assistance on the following issue:

I am working in a group developing NMS application which is deployed on JBoss 4.2 GA.

The application is developed in EJB 3.0 and Spring and has REST interfaced (JAX-RS implemented by RESTEasy).

Lately we are facing, in some scenarios, a situation in which the application server become unresponsive.

Investigating the Java thread dump I noticed that a thread is stuck “forever” in WAITING state due to  waiting in org.jboss.mx.loading.UnifiedClassLoader3.

Moreover – the dump show that the thread is waiting in RepositoryClassLoader to a lock (UnifiedClassLoader3) that it actually own (!).

PLEASE SEE BELOW FOR MORE DETAILS.

My questions:

1. Is there any  known issue of deadlock in JBoss 4.2 GA classloading mechanism(org.jboss.mx.loading.UnifiedClassLoader3)]
2. If so – was it fixed in higher JBoss/WildFly versions?
3. Is there any workaround (beside killing the application / thread) ?

Here is the dump of the relevant thread in the first point in time when is become in state “waiting” (note that the thread is locked and waiting on the same key id) :

"pool-12-thread-916" prio=10 tid=0x00007f62b83d8000 nid=0x2cc2 in Object.wait() [0x00007f62a7151000]

java.lang.Thread.State: WAITING (on object monitor)

                at java.lang.Object.wait(Native Method)

                - waiting on <0x00000007a18a7690> (a org.jboss.mx.loading.UnifiedClassLoader3)

                at java.lang.Object.wait(Object.java:503)

                at org.jboss.mx.loading.RepositoryClassLoader.loadClassImpl(RepositoryClassLoader.java:492)

                - locked <0x00000007a18a7690> (a org.jboss.mx.loading.UnifiedClassLoader3)

                at org.jboss.mx.loading.RepositoryClassLoader.loadClass(RepositoryClassLoader.java:408)

                at java.lang.ClassLoader.loadClass(ClassLoader.java:412)

                - locked <0x00000007a8c486d0> (a java.lang.Object)

                at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789)

                at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

                at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1345)

                at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1204)

                at org.springframework.util.ClassUtils.forName(ClassUtils.java:249)

Here is the same thread state after couple of minutes (the thread stay in this state till the application is killed):

pool-12-thread-916" prio=10 tid=0x00007f62b83d8000 nid=0x2cc2 in Object.wait() [0x00007f62a7151000]

java.lang.Thread.State: WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

at java.lang.Object.wait(Object.java:503)

at org.jboss.mx.loading.RepositoryClassLoader.loadClassImpl(RepositoryClassLoader.java:492)

- locked <0x00000007a1899c18> (a org.jboss.mx.loading.UnifiedClassLoader3)

at org.jboss.mx.loading.RepositoryClassLoader.loadClass(RepositoryClassLoader.java:408)

at java.lang.ClassLoader.loadClass(ClassLoader.java:412)

- locked <0x00000007a8a0f360> (a java.lang.Object)

at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1345)

at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1204)

at org.springframework.util.ClassUtils.forName(ClassUtils.java:249)

at org.springframework.util.ClassUtils.isPresent(ClassUtils.java:326)

at org.springframework.http.converter.json.Jackson2ObjectMapperBuilder.registerWellKnownModulesIfAvailable(Jackson2ObjectMapperBuilder.java:559)

at org.springframework.http.converter.json.Jackson2ObjectMapperBuilder.configure(Jackson2ObjectMapperBuilder.java:490)

at org.springframework.http.converter.json.Jackson2ObjectMapperBuilder.build(Jackson2ObjectMapperBuilder.java:434) at org.springframework.http.converter.json.MappingJackson2HttpMessageConverter.<init>(MappingJackson2HttpMessageConverter.java:56)

at org.springframework.web.client.RestTemplate.<init>(RestTemplate.java:162)

at org.springframework.web.client.RestTemplate.<init>(RestTemplate.java:176)