1 2 3 Previous Next 36 Replies Latest reply: Jan 14, 2009 3:06 PM by Kabir Khan RSS

Reimplementing ClassPools for AS

Kabir Khan Master

I'm going to bite the bullet and attempt to reimplement how the JBoss ClassPools work. I am currently working on pre-OSGi with classloaders organised by domains. Although I doubt I will have time to implement the OSGi part before AS 5.0.0.GA I would at least like to add the correct properties to my classes at this stage. A few questions

*I seem to mention you saying that if using OSGi, the classloaders are typically in the same domain with correct import/export rules set?
*How can I access all the rules for import/export for a (sub-)deployment from a deployer? That should give me the information I need about the available options.

From dev list:


On 7 Oct 2008, at 17:25, Adrian Brock wrote:


On Tue, 2008-10-07 at 16:56 +0100, Kabir Khan wrote:
Finally, this still hasn't been updated to understand
how subdeployment classloaders work. It should be asking
whether the (sub-)deployment has an
org.jboss.classloading.spi.dependency.Module
attachment and hence its own classloader.

Just to be clear, can there be arbitrary levels of classloaders within
a deployment's sub-deployments now?


That's really two different questions/issues.

The old HeirarchicalLoaderRepository only supported one level
of hierarchy, but it could only be used with top level deployments.

The war classloader was a special case, hence all the hacks
you had to do. :-)

Now the ClassLoaderDomains can go to abitrary levels,
and Subdeployment classloaders are no longer restricted
to just wars.
Subdeployment classloaders are created in their own domains.

You can determine the rules from the Module attachments
of the deployment structure.

domain/parent -> result

1) DefaultDomain/Nothing -> this is in the default domain
with a parent of the jboss bootstrap classloader

2) Domain1/Domain2 -> in domain1 with parent domain2
This applies to both top level and subdeployment classloaders.
i.e. if a subdeployment specifies a parent domain, it is
not a child of the top level deployment classloader

3) Domain1/Nothing -> A subdeployment classloader in domain1
with a parent domain equal to the subdeployment's parent
deployment's classloader (phew! :-).

The last one is harder to explain in English, here's an example
which should make it clearer.

Suppose you have an ear in Domain1, which has
a parent domain of DefaultDomain.
Inside the ear is a war in Domain2.

Domain2 is not directly a child of Domain1 like (2). Instead it goes
through the ear's classloader. That way it can see
unexported classes in the ear that others in Domain1 might
not be able to see (this is the OSGi stuff again :-).

NOTE: We may in future support (3) for top level deployments.
i.e. you could have multiple root domains for your classloaders.
That case would be similar to (3) except now the parent
is the jboss bootstrap classloader like (1)


  • 1. Re: Reimplementing ClassPools for AS
    Adrian Brock Master

     

    "kabir.khan@jboss.com" wrote:
    I'm going to bite the bullet and attempt to reimplement how the JBoss ClassPools work. I am currently working on pre-OSGi with classloaders organised by domains. Although I doubt I will have time to implement the OSGi part before AS 5.0.0.GA I would at least like to add the correct properties to my classes at this stage. A few questions

    *I seem to mention you saying that if using OSGi, the classloaders are typically in the same domain with correct import/export rules set?
    *How can I access all the rules for import/export for a (sub-)deployment from a deployer? That should give me the information I need about the available options.


    The requested rules are on the Module, e.g. importAll, exportAll and
    requirements/capabilities

    What they actually get resolved to will only be available on the DependencyInfo
    (which is really an implementation detail),
    or by looking at the actual ClassLoaderPolicy used to create the classloader,
    e.g getPackageNames() and getDelegates().

  • 2. Re: Reimplementing ClassPools for AS
    Kabir Khan Master

     

    "adrian@jboss.org" wrote:

    The requested rules are on the Module, e.g. importAll, exportAll and requirements/capabilities

    So to get this straight I can get the exported packages from the ExportedPackages(PackageCapability) of the Modules's capabilites and the imported packages from the OptionalPackages(PackageRequirement/UsesPackageRequirement)?
    Will these only be set if importAll and exportAll are false?

    What is the difference between PackageRequirement and UsesPackageRequirement? The method name is getOptionalPackageNames(). What does that mean? I also see that AbstractRequirement has an optional flag. If false, does that mean it is required? If that is the case, I don't really get what an optional import means.

    "adrian@jboss.org" wrote:

    or by looking at the actual ClassLoaderPolicy used to create the classloader,
    e.g getPackageNames() and getDelegates().

    Are you saying I should really be using this instead of Module? How do I obtain this from a deployer?

    What about version ranges? How are they represented?

    What I am doing is implementing one DelegatingClassPool per classloader, and having that backed by a ClassPoolDomain which corresponds to a classloading domain. If a CtClass cannot be found in a DelegatingClassPool it will delegate to the domain which will then look in the registered pools.

    Should things like parentFirst be handled on pool or domain level? I originally had that on the domain, but from what I gather the other settings importAll/exportAll/imports/exports should be handled on the pool level.

  • 3. Re: Reimplementing ClassPools for AS
    Adrian Brock Master

     

    "kabir.khan@jboss.com" wrote:
    "adrian@jboss.org" wrote:

    The requested rules are on the Module, e.g. importAll, exportAll and requirements/capabilities

    So to get this straight I can get the exported packages from the ExportedPackages(PackageCapability) of the Modules's capabilites and the imported packages from the OptionalPackages(PackageRequirement/UsesPackageRequirement)?
    Will these only be set if importAll and exportAll are false?


    The exported packages have nothing to do with importAll.
    exportAll is just a convenience mechanism where you don't have to explicitly
    list all package capabilities.


    What is the difference between PackageRequirement and UsesPackageRequirement? The method name is getOptionalPackageNames(). What does that mean? I also see that AbstractRequirement has an optional flag. If false, does that mean it is required? If that is the case, I don't really get what an optional import means.


    Uses = Package + optional, see discussions in the MC forum for more details
    some of it is explained in the xsd.


    "adrian@jboss.org" wrote:

    or by looking at the actual ClassLoaderPolicy used to create the classloader,
    e.g getPackageNames() and getDelegates().

    Are you saying I should really be using this instead of Module? How do I obtain this from a deployer?

    What about version ranges? How are they represented?


    In the requirement.


    What I am doing is implementing one DelegatingClassPool per classloader, and having that backed by a ClassPoolDomain which corresponds to a classloading domain. If a CtClass cannot be found in a DelegatingClassPool it will delegate to the domain which will then look in the registered pools.

    Should things like parentFirst be handled on pool or domain level? I originally had that on the domain, but from what I gather the other settings importAll/exportAll/imports/exports should be handled on the pool level.


    imports/exports are per classloader, parentFirst/parentPolicy is per ClassLoaderDomain

  • 4. Re: Reimplementing ClassPools for AS
    Adrian Brock Master

    By the way, you should consider what will make things easier for you
    as changes to the classloader api.

    Although as you know I don't agree with using the classloader to determine
    what aspects apply, when I looked at it before, the basic information you are trying
    to work out is:

    "from this classloader where does it load this other class?"
    Then from that found class you go
    class -> classloader -> application/sub-deployment -> aspect config

    The basics of this query already exists at the jmx level,
    look at the jboss.classloader on the jmx console or BaseClassLoaderMBean

     /**
     * Find the classloader for a class
     *
     * @param name the class name
     * @return the classloader or null if it is not loaded by a managed classloader
     * @throws ClassNotFoundException when the class is not found
     */
     ObjectName findClassLoaderForClass(String name) throws ClassNotFoundException;
    


    I wouldn't have a problem with adding something like the following to the Module
    ClassLoader findClassLoaderForClass(String name) throws ClassNotFoundException;
    or
    Module findModuleForClass(String name) throws ClassNotFoundException;
    or other things that might be useful.

    Once you have the classloader/module you can work out the
    (sub-)deployment/aspect domain.

    Or equally if that is not useful to you, we can discuss other helpers that are. ;-)


  • 5. Re: Reimplementing ClassPools for AS
    Kabir Khan Master

    The work so far has been commited against
    https://jira.jboss.org/jira/browse/JBAOP-666

    I've had to look at other things so far this week, so unfortunately I don't think this will be done before the GA cutoff point on Monday. I will try to make the existing code work for Adrian's original comments.

  • 6. Re: Reimplementing ClassPools for AS
    Kabir Khan Master

    I started off replicating the AS 4 classloaders with my new classpools, and stumbled upon the following inconsistency when trying out how they work. Does anybody have any pointers?

     public void testUndeployParentDomainClassLoader() throws Exception
     {
     ClassLoader globalA = null;
     ClassLoader globalB = null;
     ClassLoader child = null;
     try
     {
     try
     {
     globalA = createGlobalClassLoader(JAR_A_1); //Contains CLASS_A
     assertCannotLoadClass(globalA, CLASS_B);
    
     child = createChildClassLoader(JAR_C_1, true); //Contains CLASS_C
     assertCannotLoadClass(child, CLASS_B);
    
     globalB = createGlobalClassLoader(JAR_B_1); //Contains CLASS_B
     Class<?> bFromChild = child.loadClass(CLASS_B);
     Class<?> bFromA = globalA.loadClass(CLASS_B);
     assertSame(globalB, bFromA.getClassLoader());
     assertSame(bFromA, bFromChild);
     }
     finally
     {
     removeClassLoaderFromRepository(globalB); //This should remove the classloader containing JAR_B_1/CLASS_B
     }
     assertCannotLoadClass(child, CLASS_B); //Fails since we can still find CLASS_B
     }
     finally
     {
     removeClassLoaderFromRepository(globalA);
     }
     }
    

    If I try the same from sibling loaders it works as expected
     public void testUndeploySibling() throws Exception
     {
     ClassLoader clA = null;
     ClassLoader clB = null;
     try
     {
     try
     {
     clA = createGlobalClassLoader(JAR_A_1);
     assertCannotLoadClass(clA, CLASS_B);
    
     clB = createGlobalClassLoader(JAR_B_1);
     Class<?> bFromA = clA.loadClass(CLASS_B);
     assertSame(clB, bFromA.getClassLoader());
     }
     finally
     {
     removeClassLoaderFromRepository(clB);
     }
     assertCannotLoadClass(clA, CLASS_B);
     }
     finally
     {
     removeClassLoaderFromRepository(clA);
     }
     }
    


    The helper functions are here:
     protected ClassLoader createGlobalClassLoader(URL url) throws Exception
     {
     ClassLoader cl = globalRepository.newClassLoader(url, true); //This is the main repository
     registeredClassLoaders.add(cl);
     return cl;
     }
    
     protected ClassLoader createChildClassLoader(URL url, boolean parentFirst) throws Exception
     {
     HeirarchicalLoaderRepository3 repository = new HeirarchicalLoaderRepository3(getServer(), MAIN_LOADER_REPOSITORY_OBJECT_NAME);
     repository.setUseParentFirst(parentFirst);
     ClassLoader cl = repository.newClassLoader(url, true);
     registeredClassLoaders.add(cl);
     return cl;
     }
    
     protected void removeClassLoaderFromRepository(ClassLoader cl)
     {
     if (cl != null)
     {
     if (cl instanceof RepositoryClassLoader)
     {
     LoaderRepository repository = ((RepositoryClassLoader)cl).getLoaderRepository();
     repository.removeClassLoader(cl);
     }
     }
     }
    
     protected void assertCannotLoadClass(ClassLoader cl, String className)
     {
     try
     {
     cl.loadClass(className);
     fail("Should not have been able to load " + className);
     }
     catch(Exception expected)
     {
     }
     }
    


  • 7. Re: Reimplementing ClassPools for AS
    Kabir Khan Master

    Adrian,

    I have started looking at the new classloaders. Before I do any OSGi stuff, I would like to first replicate the behaviour currently used out-of-the-box in AS, i.e. import/exportAll with domains. To help my understanding, which tests in the jboss-cl project do this?

    After an initial look these look likely:
    org.jboss.test.classloader.domain.test.*
    org.jboss.test.classloading.dependency.test.DomainUnitTestCase
    org.jboss.test.classloading.dependency.test.HierarchicalDomainUnitTestCase
    org.jboss.test.classloading.vfs.metadata.test.DomainUnitTestCase

    Are there any others I should look at at this stage?

  • 8. Re: Reimplementing ClassPools for AS
    Adrian Brock Master

    The osgi stuff is anything that does not use import-all=true, export-all
    or uses delegates/requirements/capabilities

    I don't think you'll find tests that only test legacy rules (except possibliy the "old" tests).

    In general the tests are aimed at testing an area (e.g. different configs of parent
    domains) rather than a single use-case (e.g. there are no import-all=true tests
    they are mixed in with import-all=false tests in the same area, e.g. loading resources).

  • 9. Re: Reimplementing ClassPools for AS
    Kabir Khan Master

    Adrian,

    If I register a ClassLoaderPolicy that has a parent loader, I end up with a BaseClassLoader with no parent loader and an underlying ClassLoaderDomain that has a ClassLoaderToLoaderAdapter as its parent. The CLTLA contains the parent loader of the ClassLoaderPolicy. I need to be able to access what the parent loader is when constructing the pools, but don't see a way to get hold of that using the Loader interface returned by ClassLoaderDomain. Trying to hack it and use CLTLA directly also does not allow me access to the underlying parent loader.

    Is there a way to get hold of this from the ClassLoaderSystem or elsewhere?

  • 10. Re: Reimplementing ClassPools for AS
    Adrian Brock Master

    I've said this before, but I'll repeat it.

    You shouldn't be trying to use the ClassLoaderSystem api for this.

    The information you want is available in the ClassLoading/Modules api in the form
    of what the parent domain is.

    If its a sub-deployment with a Module then the parent is the classloader of the
    parent deployment. (If it has no explicit parent domain specified.)
    In all other cases it is the getDeterminedParentDomainName()

    This last part is kind of an implicit rule created by the deployers
    since only they understand what a subdeployment actually is. :-)

    Like I said before, if we need to make something more explicit in the Module api
    or some helper class to make your life easier then we can do that.

    The current "implicit" rule is here:
    http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbossas/projects/jboss-deployers/trunk/deployers-impl/src/main/java/org/jboss/deployers/plugins/classloading/AbstractLevelClassLoaderSystemDeployer.java?revision=78007&view=markup

    Although the first comment in the code is incomplete/wrong, I've corrected here.

     if (unit.isTopLevel() || module.getParentDomainName() != null)
     {
     // Top level or subdeployment with a parent domain, just create the classloader
     return classLoaderPolicyModule.registerClassLoaderPolicy(system);
     }
     else
     {
     // Subdeployment that wants a classloader
     ClassLoader parentClassLoader = unit.getParent().getClassLoader();
     return classLoaderPolicyModule.registerClassLoaderPolicy(system, parentClassLoader);
    


  • 11. Re: Reimplementing ClassPools for AS
    Kabir Khan Master

    I am trying to replicate this with my tests, although the case that I am trying to do is probably not something that will happen in the real world (i.e. in AS)?

    I am basically creating a URLClassLoader, and then registering a policy containing a scoped child domain as its child. I then cannot get hold of the parent URLClassLoader as mentioned.

    In AS I also don't see any way to get hold of the parent classloader from the deployment units, although in AS I belive the following to be true.
    *The default domain is fixed up to use the loader of BaseClassLoaderDomain as its parent, which in AS means the NoAnnotationURLClassLoader.
    *Other domains will always be children of either the default domain or an intermediate domain.

    So my test is probably a bit too ambitious?

  • 12. Re: Reimplementing ClassPools for AS
    Kabir Khan Master

    So I either get the parent domain name from the Module and get hold of the domain using ClassLoaderSystem.getDomain() OR the parent domain name from Module is null in which case I can guess that the parent is the loader of the BaseClassLoaderDomain class?

  • 13. Re: Reimplementing ClassPools for AS
    Kabir Khan Master

    Adrian/Ales,

    I've had a look at http://www.jboss.com/index.html?module=bb&op=viewtopic&t=130122, but it is not clear to me what the difference is between ModuleRequirement and PackageRequirement? Does a module represent a jar, thus meaning import everything from that jar? And is package more fine-grained, meaning only import these packages from that jar?

    Whatever the answer, I suppose the Module/Package difference would also apply to the capabilities?

    Also, what is UsesPackageRequirement?

  • 14. Re: Reimplementing ClassPools for AS
    Ales Justin Master

     

    "kabir.khan@jboss.com" wrote:

    Does a module represent a jar, thus meaning import everything from that jar?

    I wouldn't call it 'import everything from that bundle'.
    It's more like a plain bundle dependency, where you trust that bundle to have the packages/classes you need.
    e.g. you know you're using Hibernate, so instead of doing every Hibernate import,
    you just say I need Hibernate module, as you know that the Hibernate stuff you use it's gonna be there

    "kabir.khan@jboss.com" wrote:
    And is package more fine-grained, meaning only import these packages from that jar?

    Yup, exactly.
    It's when you need more fine-grained dependencies.

    "kabir.khan@jboss.com" wrote:

    Whatever the answer, I suppose the Module/Package difference would also apply to the capabilities?

    Module capabilities are filtered.
    e.g. VFSDeploymentClassLoaderPolicyModule
     // We have a module capability
     Object version = getVersion();
     Capability capability = factory.createModule(getName(), version);
     capabilities.add(capability);
    
     // Do we determine package capabilities
     ClassFilter included = getIncluded();
     ClassFilter excluded = getExcluded();
     ClassFilter excludedExport = getExcludedExport();
     ExportAll exportAll = getExportAll();
     if (exportAll != null)
     {
     Set<String> exportedPackages = PackageVisitor.determineAllPackages(roots, excludedRoots, exportAll, included, excluded, excludedExport);
     for (String packageName : exportedPackages)
     {
     capability = factory.createPackage(packageName, version);
     capabilities.add(capability);
     }
     }
    
     return capabilities;
    


    Package capabilities would normally come from jboss-classloading.xml,
    an explicit definition -> capabilities/package/name+version ...

    "kabir.khan@jboss.com" wrote:

    Also, what is UsesPackageRequirement?

    It's when some export depends on some other bundle/api.
    A comma-separated list of package names that are used by the
    exported package. Note that the use of a comma in the value requires it
    to be enclosed in double quotes. If this exported package is chosen as an
    export, then the resolver must ensure that importers of this package wire
    to the same versions of the package in this list.
    
    ...
    
    Classes can depend on classes in other packages. For example, when they
    extend classes from another package, or these other classes appear in
    method signatures. It can therefore be said that a package uses other packages.
    These inter-package dependencies are modeled with the uses directive
    on the Export-Package header.
    For example, org.osgi.service.http depends on the package javax.servlet
    because it is used in the API. The export definition of the
    org.osgi.service.http must therefore contain the uses directive with the
    javax.servlet package as its value.
    

    So you need to import that, in order to resolve properly,
    as 'uses' enforces a transitive dependency on exact version.

1 2 3 Previous Next