JBoss ClassLoader History

JBoss ClassLoader History

 

This document describes the evolution of the JBoss ClassLoader, including some motivation for the changes and provides some interesting asides on how (some would say unnecessarily) difficult it can be to implement your own classloader.

 

JBoss-2.x - the early years and the first experiences with hot deployment

 

JBoss was the first application server to have hot deployment for all user level components, previously servlet containers had let you change servlet classes but JBoss extended this feature to ejb containers and resource adapters.

 

For system level classes a ClassPathExtension MBean was available which actually just modified the java.class.path system variable. So in reality you could just modify the JVM's classpath, the MBean made the configuration easier and more persistent.

 

Hot deployment was implemented by isolating the replacable classes in a seperate classloader that is thrown away and recreated when the classes change. Classes in the class path cannot be hot deployed since there is no way to recreate that classloader (it is under the control of the JRE).

 

More recently an agent can be used to redefine classes with some fairly restrictive rules on what can be changed.

 

With the hierarchical nature of java classloaders, having a seperate classloader for some classes makes it difficult to share classes. If for example you have two ejb jars (not in the same ear) with seperate classloaders they won't be able to see each others classes. This leads to the famous ClassCastException problems where each classloader has its own copy of the same class (under java rules they are not the same class even though they have the same name). If they try to communicate using the more performant call by reference semantics you will get a problem.

In principle this shouldn't have been a problem since the EJB spec at the time only supported call by value semantics of remote interfaces, but JBoss had another new feature that would later get included in the spec as local interfaces.

 

Even where the classes are visible between classloaders, package private access doesn't work if the classes are from different classloaders.

 

Interestingly, the dynamic loading feature of servlets mentioned above is now deprecated in the spec. You typically have to do extra work to enable it, assuming the container still supports it.

 

JBoss-3.x/4.x - UnifiedClassLoader and peer classloader models

 

JBoss version 3 set out to solve some of these problems. It was actually more ambitous in wanting to make the server implementation hot deployable as well, introducing the .sar archive.JBossClassLoadingUseCases Instead here we will just describe the major changes and the issues discovered.

 

The biggest change was the introduction of a peer classloading implementation. Unlike the default hierarchical model of the JRE this resolves the visibility issue. JBoss was not the first to try to do this, for example the OSGi spec (and a number of other more experimental projects) had tried to do this, but JBoss was the first to do it in real anger. This meant it was the first to discover a number of serious concurrency issues within the JRE itself.

 

The first major problem was the deadlocking problem. Classloading rules requires that two classloading requests on the same classloader are not executed concurrently which requires some kind of locking. In a peer delegation model this could lead to deadlock issues if not done correctly. This proved to be more difficult than it should have been because the JRE enforces its own locking in an undocumented way which led to the most popular java bug of all time. Even after working around that issue, further concurrency issues were found deeper inside the JRE. The second issue was resolved by a complete rewrite of the class validation code inside the JRE, the first issue remains unresolved at the time of writing, although there are plans for Java 7 to resolve it.

 

JBoss couldn't wait for those bugs to be resolved, instead the UnifiedClassLoader works around them at the cost of some extra complexity.

 

The original UnifiedClassLoader was implemented as a single repository of peer classloaders. Each classloader can see the classes of each other classloader by delegating requests through the repository. Later a HeirarchicalLoaderRepository (sic!) was introduced which allowed the reintroduction of classloader isolation for those that wanted it. The fact that this was a later addition shows up as some ugliness in its implementation details.

 

Another change along the way was that the original UnifiedClassLoader created one classloader per jar file. This proved to be inefficient and also suffered from the package private problem. This was soon changed to have one classloader per top level deployment.

 

You can register other classloader implementations with the repository but they do not fully take part in the peer delegation.The UnifiedClassLoaders can use them as full peers but not vice-versa.

 

Initially, web applications also used a UnifiedClassLoader for classloading, but this was changed by default to use the Tomcat classloader with a configuration option to choose which model you want. Most web developers expected their web applications to be isolated from other classloaders.

 

One of the major outstanding issues not resolved with the UnifiedClassLoader is classloading dependencies. The idea being that if one application (A) uses the classes of another application (B), the system should know to redeploy A when B gets redeployed, otherwise it will be referencing stale classes. There were actually two different attempts to try to make this work without the user having to configure anything. Neither attempt really worked and both were dropped.

 

JBoss-5.x - The New ClassLoader

 

JBoss ClassLoader Introduction is explained more fully elsewhere on this WIKI. Like the UnifiedClassLoader, its aim is to try to resolve some of the problems discovered using the old implementation and some additional new requirements. Unlike the UnifiedClassLoader the new classloader is fully backwards compatible, JBossAS 5.0.x Changes FAQThat is, all the classloading configurations from JBoss3/4 will still work with the new implementation. In fact, extra care has been taken to ensure it is configured to behave like the older version by default, although some of the default configurations have moved with the re-implementation of the deployers.

 

The new classloader shares much of the same design and implementation details with the UnifiedClassLoader but makes some improvements on it.

  • It is no longer dependent upon JMX so it can be used standalone in any environment
  • You can implement your own classloader policy without having to know the gory details and workarounds that are required to implement a peer classloader delegation model
  • You have a lot more control on what other classloaders your classloader delegates to
  • You have more control over which classes other classloaders can see
  • The hierarchical repositories have been reimplemented more cleanly as domains and can now extend beyond more than one level

 

The default classloader policy uses the new JBoss VFS to resolve resources. A major motivation for this change is to move the handling of the file locking problems (seen on Windows) or file handle leaks (seen on Linux) into one place. Previously the UnifiedClassLoader extended the URLClassLoader which tends to leak jar file references if you are not careful, a problem Sun refuse to fix. The other motivation is that it is impossible to represent a resource in an embedded jar using a URL meaning you have to redundantly extract the jar into a temporary location.

 

On top of the new classloader implementation is a new layer that introduces classloading dependencies and adds helper utilities such as scanning for class annotations. This layer includes all the features you can find in OSGi including versioning, along with the ability to "write your own dependency" should you need something more complicated.

 

The new layer uses a "metamodel" so you don't need to know the implementation details of the classloader. The older classloader configurations are automatically mapped into this metamodel or you can use it more directly using a META-INF/jboss-classloading.xml. Once our OSGi implementation is ready for prime time you will also be able to use the OSGi manifest headers to define your classloading dependencies.

 

Finally, the classloader implementation is actually just a choice made in the new JBoss deployer infrastructure. In principle (though I wouldn't recommend it to non-experts) it can be swapped out for any classloader implementation of you choice. Assuming the new classloader implementation can replicate all the features that you need.

 

The Future

 

This section is of course speculative.

 

Java7 should introduce a couple of new features to classloading that are interesting.

  • The above mentioned abillity to do proper concurrent classloading which will remove the locking problems and workarounds
  • The "super package" modularization of JSR294

 

We can also expect (as in the past) that as we gain more experience with how the new features of the classloader are used this will lead to further improvements in the classloader implementation.