Jboss AOP classloading bug
I'm using JBoss 6.0.0 with AOP for logging user actions and function call durations. Functions of interest are marked by our custom @Trace annotation and there is an our tracing aspect, that do all the work. But sometimes, probably after jboss restart, some random functionality of the project becomes unavailable, e.g. some dropdown lists didn't working or some buttons pressing do nothing. After second rebooting everething work fine. I can't reproduce this on my machine, but customers regulary (once or twice a month) reporting about the problem. Here is the typical log with org.jboss.classpool logging on:
When all is functioning properly the log look's like:
I have made some investigation and degugging, so probably I have found the cause of the problem.
At first, if you look at ClassPoolToDomainAdapter with a lot of byte array class paths in it, you could see the difference, at normal working this list
contains BaseClassLoader@xxxxxx{vfs:///D:/retailHub/loan/server/default/deploy/myProject.ear} ClassPath at the end, but then the problem is it doesn't.
After deepest code investigation I figured out, that this could happen then something call close() on а ClassPool, and this probably happens inside unregisterClassLoader() function.
During joint point constraction Aspect Manager calls several times unregisterClassLoader() function:
ClassPoolRepository
public void unregisterClassLoader(ClassLoader classLoader)
{
registryHandler.unregisterClassLoader(classLoader); // <-- calls JBossClRegistryHandler.unregisterClassLoader()
if (callbacks != null && callbacks.size() > 0)
{
for (ClassPoolRepositoryCallback callback : callbacks)
{
callback.classLoaderUnregistered(classLoader);
}
}
}
which in turn calls JBossClRegistryHandler.registerClassLoader() functions with possible result of a closing the class pool.
JBossClRegistryHandler
public ClassPool registerClassLoader(ClassLoader classLoader)
{
ScopedClassPool classPool = (ScopedClassPool) successor.registerClassLoader(classLoader);
if (classPool == null) // <-- true
{
// TODO check this works; was delegate before
successor.unregisterClassLoader(classLoader); // <-- calls classLoader.close()
}
else
{
Module module = getModuleForClassLoader(classLoader);
this.registeredModules.put(module, classPool);
}
return classPool;
}
I can't understand what is the purpose of the DefaultClassLoaderRegistryHandler, but it can return null from registerClassLoader() function,
probaly then several jointpoints are generated concurrently from different threads:
ClassPoolRepository$DefaultClassLoaderRegistryHandler
public ClassPool registerClassLoader(ClassLoader classLoader)
{
if (classLoader == null)
{
classLoader = SecurityActions.getContextClassLoader();
}
if (currentClassLoaders.putIfAbsent(classLoader, Boolean.TRUE) != null)
{
return null; // <-- Returns null
}
ScopedClassPool classPool = (ScopedClassPool) ClassPoolRepository.super.registerClassLoader(classLoader);
currentClassLoaders.remove(classLoader);
return classPool;
}
ScopedClassPoolRepositoryImpl:
public void unregisterClassLoader(ClassLoader cl) {
synchronized (registeredCLs) {
ScopedClassPool pool = registeredCLs.remove(cl);
if (pool != null)
pool.close(); // <-- close
}
}
ScopedClassPool:
public void close() {
this.removeClassPath(classPath); // <-- remove BaseClassLoader@xxxxxx{vfs:///D:/retailHub/loan/server/default/deploy/myProject.ear} classpath
classPath.close();
classes.clear();
softcache.clear();
}
During joint point creation the myProject.war class loader is used, it redirects calls to JBossClClassPoolDomain, and in turn to SuperClassesFirstWeavingStrategy, and finally
the reference to the old ClassPool (with a lot of byte array class paths) is used. Is has no myProject.ear ClassLoader any more, so NotFoundException is fired.
I don't know how to fix this, I will appreciate your help.
