This is the class blacklist. I researched this back when we were on JBoss 4.2.3 and now I'm revisiting it for JBoss 5.1.0, since things seem to have changed.
Hibernate calls class.forName() on every token in each unique query. This is one of many reasons why it's crucially important to use bind parameters in your queries. If you are using bind params, your query is parsed by Hibernate once, then placed in some parsed query cache. This parsing is what calls Class.forName() on each token in the query. If you're not using bind params, each query looks unique, and it gets reparsed each time the aruments in the WHERE clause change. The endless stream of unique arguments in the where clause (from your 100k SOAP requests) end up as parameters to Class.forName(). JBoss's classloader caches failed class lookups, as a performance enhancement. In JBoss 4.2.3, this was done in RepositoryClassLoader. It was possible to set two properties which would not stop the accumulation in the class blacklist, but would stop the OutOfMemoryErrors from occurring by allowing the JVM to cull the blacklist when memory gets tight.
However, in JBoss 5.1.0, (probably 5.x on), RepositoryClassLoader appears to be unused (zero instances in my heap dump today). The class blacklist is now contained in BaseClassLoader, which does not have the same runaway allocation guard, as far as I can tell.
You might try filing a JIRA since this seems like a regression, but Adrian Brock has already weighed-in on the topic once before, and he's of the opinion that it's the framework's problem, not JBoss'. "There are many "stupid" webapp frameworks (and other code) around that try to continually load non-existant classes." https://jira.jboss.org/browse/JBAS-3041
In short, make sure that each and every one of your queries uses bind params, for this reason and numerous others all related to increased performance and security.
Thank you for your kind answer.
The queries are generated by jpacriteria library, another layer over hibernate.
I found out that each object is given an alias, for example:
SELECT $obj$5cdba0 FROM com.example.MyEntity $obj$5cdba0
Actually the query does work, but makes the class loader blacklist the String "$obj$5cdba0.class" because someone (hibernate?) is trying to forName it.
I see these solutions, in order of dirtyness(?):
1) patch jpacriteria library in order to avoid generating insane queries
2) disable blacklisting by using jboss-classloading.xml (quick, any drawbacks?)
3) clear blacklist every day with a quartz job, by using jmx
Regarding 2) there are 104 classloader instances, but I should override the blacklist policy only for the classloader with id="vfsfile:$JBOSS_HOME/server/node1/conf/jboss-service.xml". Where should I put the jboss-classloading.xml?
Any other ideas?
(Someone tell Adrian Brock that jpacriteria should be added to the *blacklist* of stupid frameworks
And btw why is the class loader using an unlimited cache for a blacklist?? Why not LRU?)
Yes, you might use jconsole (bundled since Java6).
Here you have a view to the running JVM and invocation of GC.
Local you need only the PID, remote you must enable the access via JVM option.
The JConsole watch the given JVM, but if you observe too long you affect the application because some (statistic) data will be stored in the observed JVM.
Also an explicit call of GC should only be done for test, production enviroment should run without.
When i analyzed memory leak problem through Eclipse Memory Analyzer tool. I see that most of the memory leak is caused by
org.jboss.classloader.spi.base.BaseClassLoader as mentioned below.
org.jboss.classloader.spi.base.BaseClassLoader @ 0x7021143c8 37,662 2,886,184 1,020,624,664 78.02% <system class loader> 74,919 5,242,240 81,034,736 6.19% org.jboss.bootstrap.NoAnnotationURLClassLoader @ 0x70001a228 18,873 1,854,128 79,102,152 6.05% org.jboss.classloader.spi.base.BaseClassLoader @ 0x70210ed38 3,838 77,080 50,101,280 3.83% org.jboss.classloader.spi.base.BaseClassLoader @ 0x70001a1a0 471 42,040 47,995,688 3.67%
I wanted to know how baseClassLoader is causing a memory problem.
Thanks in Advance.