1 2 3 Previous Next 36 Replies Latest reply on Jan 14, 2009 3:06 PM by kabirkhan

    Reimplementing ClassPools for AS

    kabirkhan

      I'm going to bite the bullet and attempt to reimplement how the JBoss ClassPools work. I am currently working on pre-OSGi with classloaders organised by domains. Although I doubt I will have time to implement the OSGi part before AS 5.0.0.GA I would at least like to add the correct properties to my classes at this stage. A few questions

      *I seem to mention you saying that if using OSGi, the classloaders are typically in the same domain with correct import/export rules set?
      *How can I access all the rules for import/export for a (sub-)deployment from a deployer? That should give me the information I need about the available options.

      From dev list:


      On 7 Oct 2008, at 17:25, Adrian Brock wrote:


      On Tue, 2008-10-07 at 16:56 +0100, Kabir Khan wrote:
      Finally, this still hasn't been updated to understand
      how subdeployment classloaders work. It should be asking
      whether the (sub-)deployment has an
      org.jboss.classloading.spi.dependency.Module
      attachment and hence its own classloader.

      Just to be clear, can there be arbitrary levels of classloaders within
      a deployment's sub-deployments now?


      That's really two different questions/issues.

      The old HeirarchicalLoaderRepository only supported one level
      of hierarchy, but it could only be used with top level deployments.

      The war classloader was a special case, hence all the hacks
      you had to do. :-)

      Now the ClassLoaderDomains can go to abitrary levels,
      and Subdeployment classloaders are no longer restricted
      to just wars.
      Subdeployment classloaders are created in their own domains.

      You can determine the rules from the Module attachments
      of the deployment structure.

      domain/parent -> result

      1) DefaultDomain/Nothing -> this is in the default domain
      with a parent of the jboss bootstrap classloader

      2) Domain1/Domain2 -> in domain1 with parent domain2
      This applies to both top level and subdeployment classloaders.
      i.e. if a subdeployment specifies a parent domain, it is
      not a child of the top level deployment classloader

      3) Domain1/Nothing -> A subdeployment classloader in domain1
      with a parent domain equal to the subdeployment's parent
      deployment's classloader (phew! :-).

      The last one is harder to explain in English, here's an example
      which should make it clearer.

      Suppose you have an ear in Domain1, which has
      a parent domain of DefaultDomain.
      Inside the ear is a war in Domain2.

      Domain2 is not directly a child of Domain1 like (2). Instead it goes
      through the ear's classloader. That way it can see
      unexported classes in the ear that others in Domain1 might
      not be able to see (this is the OSGi stuff again :-).

      NOTE: We may in future support (3) for top level deployments.
      i.e. you could have multiple root domains for your classloaders.
      That case would be similar to (3) except now the parent
      is the jboss bootstrap classloader like (1)


        • 1. Re: Reimplementing ClassPools for AS

           

          "kabir.khan@jboss.com" wrote:
          I'm going to bite the bullet and attempt to reimplement how the JBoss ClassPools work. I am currently working on pre-OSGi with classloaders organised by domains. Although I doubt I will have time to implement the OSGi part before AS 5.0.0.GA I would at least like to add the correct properties to my classes at this stage. A few questions

          *I seem to mention you saying that if using OSGi, the classloaders are typically in the same domain with correct import/export rules set?
          *How can I access all the rules for import/export for a (sub-)deployment from a deployer? That should give me the information I need about the available options.


          The requested rules are on the Module, e.g. importAll, exportAll and
          requirements/capabilities

          What they actually get resolved to will only be available on the DependencyInfo
          (which is really an implementation detail),
          or by looking at the actual ClassLoaderPolicy used to create the classloader,
          e.g getPackageNames() and getDelegates().

          • 2. Re: Reimplementing ClassPools for AS
            kabirkhan

             

            "adrian@jboss.org" wrote:

            The requested rules are on the Module, e.g. importAll, exportAll and requirements/capabilities

            So to get this straight I can get the exported packages from the ExportedPackages(PackageCapability) of the Modules's capabilites and the imported packages from the OptionalPackages(PackageRequirement/UsesPackageRequirement)?
            Will these only be set if importAll and exportAll are false?

            What is the difference between PackageRequirement and UsesPackageRequirement? The method name is getOptionalPackageNames(). What does that mean? I also see that AbstractRequirement has an optional flag. If false, does that mean it is required? If that is the case, I don't really get what an optional import means.

            "adrian@jboss.org" wrote:

            or by looking at the actual ClassLoaderPolicy used to create the classloader,
            e.g getPackageNames() and getDelegates().

            Are you saying I should really be using this instead of Module? How do I obtain this from a deployer?

            What about version ranges? How are they represented?

            What I am doing is implementing one DelegatingClassPool per classloader, and having that backed by a ClassPoolDomain which corresponds to a classloading domain. If a CtClass cannot be found in a DelegatingClassPool it will delegate to the domain which will then look in the registered pools.

            Should things like parentFirst be handled on pool or domain level? I originally had that on the domain, but from what I gather the other settings importAll/exportAll/imports/exports should be handled on the pool level.

            • 3. Re: Reimplementing ClassPools for AS

               

              "kabir.khan@jboss.com" wrote:
              "adrian@jboss.org" wrote:

              The requested rules are on the Module, e.g. importAll, exportAll and requirements/capabilities

              So to get this straight I can get the exported packages from the ExportedPackages(PackageCapability) of the Modules's capabilites and the imported packages from the OptionalPackages(PackageRequirement/UsesPackageRequirement)?
              Will these only be set if importAll and exportAll are false?


              The exported packages have nothing to do with importAll.
              exportAll is just a convenience mechanism where you don't have to explicitly
              list all package capabilities.


              What is the difference between PackageRequirement and UsesPackageRequirement? The method name is getOptionalPackageNames(). What does that mean? I also see that AbstractRequirement has an optional flag. If false, does that mean it is required? If that is the case, I don't really get what an optional import means.


              Uses = Package + optional, see discussions in the MC forum for more details
              some of it is explained in the xsd.


              "adrian@jboss.org" wrote:

              or by looking at the actual ClassLoaderPolicy used to create the classloader,
              e.g getPackageNames() and getDelegates().

              Are you saying I should really be using this instead of Module? How do I obtain this from a deployer?

              What about version ranges? How are they represented?


              In the requirement.


              What I am doing is implementing one DelegatingClassPool per classloader, and having that backed by a ClassPoolDomain which corresponds to a classloading domain. If a CtClass cannot be found in a DelegatingClassPool it will delegate to the domain which will then look in the registered pools.

              Should things like parentFirst be handled on pool or domain level? I originally had that on the domain, but from what I gather the other settings importAll/exportAll/imports/exports should be handled on the pool level.


              imports/exports are per classloader, parentFirst/parentPolicy is per ClassLoaderDomain

              • 4. Re: Reimplementing ClassPools for AS

                By the way, you should consider what will make things easier for you
                as changes to the classloader api.

                Although as you know I don't agree with using the classloader to determine
                what aspects apply, when I looked at it before, the basic information you are trying
                to work out is:

                "from this classloader where does it load this other class?"
                Then from that found class you go
                class -> classloader -> application/sub-deployment -> aspect config

                The basics of this query already exists at the jmx level,
                look at the jboss.classloader on the jmx console or BaseClassLoaderMBean

                 /**
                 * Find the classloader for a class
                 *
                 * @param name the class name
                 * @return the classloader or null if it is not loaded by a managed classloader
                 * @throws ClassNotFoundException when the class is not found
                 */
                 ObjectName findClassLoaderForClass(String name) throws ClassNotFoundException;
                


                I wouldn't have a problem with adding something like the following to the Module
                ClassLoader findClassLoaderForClass(String name) throws ClassNotFoundException;
                or
                Module findModuleForClass(String name) throws ClassNotFoundException;
                or other things that might be useful.

                Once you have the classloader/module you can work out the
                (sub-)deployment/aspect domain.

                Or equally if that is not useful to you, we can discuss other helpers that are. ;-)


                • 5. Re: Reimplementing ClassPools for AS
                  kabirkhan

                  The work so far has been commited against
                  https://jira.jboss.org/jira/browse/JBAOP-666

                  I've had to look at other things so far this week, so unfortunately I don't think this will be done before the GA cutoff point on Monday. I will try to make the existing code work for Adrian's original comments.

                  • 6. Re: Reimplementing ClassPools for AS
                    kabirkhan

                    I started off replicating the AS 4 classloaders with my new classpools, and stumbled upon the following inconsistency when trying out how they work. Does anybody have any pointers?

                     public void testUndeployParentDomainClassLoader() throws Exception
                     {
                     ClassLoader globalA = null;
                     ClassLoader globalB = null;
                     ClassLoader child = null;
                     try
                     {
                     try
                     {
                     globalA = createGlobalClassLoader(JAR_A_1); //Contains CLASS_A
                     assertCannotLoadClass(globalA, CLASS_B);
                    
                     child = createChildClassLoader(JAR_C_1, true); //Contains CLASS_C
                     assertCannotLoadClass(child, CLASS_B);
                    
                     globalB = createGlobalClassLoader(JAR_B_1); //Contains CLASS_B
                     Class<?> bFromChild = child.loadClass(CLASS_B);
                     Class<?> bFromA = globalA.loadClass(CLASS_B);
                     assertSame(globalB, bFromA.getClassLoader());
                     assertSame(bFromA, bFromChild);
                     }
                     finally
                     {
                     removeClassLoaderFromRepository(globalB); //This should remove the classloader containing JAR_B_1/CLASS_B
                     }
                     assertCannotLoadClass(child, CLASS_B); //Fails since we can still find CLASS_B
                     }
                     finally
                     {
                     removeClassLoaderFromRepository(globalA);
                     }
                     }
                    

                    If I try the same from sibling loaders it works as expected
                     public void testUndeploySibling() throws Exception
                     {
                     ClassLoader clA = null;
                     ClassLoader clB = null;
                     try
                     {
                     try
                     {
                     clA = createGlobalClassLoader(JAR_A_1);
                     assertCannotLoadClass(clA, CLASS_B);
                    
                     clB = createGlobalClassLoader(JAR_B_1);
                     Class<?> bFromA = clA.loadClass(CLASS_B);
                     assertSame(clB, bFromA.getClassLoader());
                     }
                     finally
                     {
                     removeClassLoaderFromRepository(clB);
                     }
                     assertCannotLoadClass(clA, CLASS_B);
                     }
                     finally
                     {
                     removeClassLoaderFromRepository(clA);
                     }
                     }
                    


                    The helper functions are here:
                     protected ClassLoader createGlobalClassLoader(URL url) throws Exception
                     {
                     ClassLoader cl = globalRepository.newClassLoader(url, true); //This is the main repository
                     registeredClassLoaders.add(cl);
                     return cl;
                     }
                    
                     protected ClassLoader createChildClassLoader(URL url, boolean parentFirst) throws Exception
                     {
                     HeirarchicalLoaderRepository3 repository = new HeirarchicalLoaderRepository3(getServer(), MAIN_LOADER_REPOSITORY_OBJECT_NAME);
                     repository.setUseParentFirst(parentFirst);
                     ClassLoader cl = repository.newClassLoader(url, true);
                     registeredClassLoaders.add(cl);
                     return cl;
                     }
                    
                     protected void removeClassLoaderFromRepository(ClassLoader cl)
                     {
                     if (cl != null)
                     {
                     if (cl instanceof RepositoryClassLoader)
                     {
                     LoaderRepository repository = ((RepositoryClassLoader)cl).getLoaderRepository();
                     repository.removeClassLoader(cl);
                     }
                     }
                     }
                    
                     protected void assertCannotLoadClass(ClassLoader cl, String className)
                     {
                     try
                     {
                     cl.loadClass(className);
                     fail("Should not have been able to load " + className);
                     }
                     catch(Exception expected)
                     {
                     }
                     }
                    


                    • 7. Re: Reimplementing ClassPools for AS
                      kabirkhan

                      Adrian,

                      I have started looking at the new classloaders. Before I do any OSGi stuff, I would like to first replicate the behaviour currently used out-of-the-box in AS, i.e. import/exportAll with domains. To help my understanding, which tests in the jboss-cl project do this?

                      After an initial look these look likely:
                      org.jboss.test.classloader.domain.test.*
                      org.jboss.test.classloading.dependency.test.DomainUnitTestCase
                      org.jboss.test.classloading.dependency.test.HierarchicalDomainUnitTestCase
                      org.jboss.test.classloading.vfs.metadata.test.DomainUnitTestCase

                      Are there any others I should look at at this stage?

                      • 8. Re: Reimplementing ClassPools for AS

                        The osgi stuff is anything that does not use import-all=true, export-all
                        or uses delegates/requirements/capabilities

                        I don't think you'll find tests that only test legacy rules (except possibliy the "old" tests).

                        In general the tests are aimed at testing an area (e.g. different configs of parent
                        domains) rather than a single use-case (e.g. there are no import-all=true tests
                        they are mixed in with import-all=false tests in the same area, e.g. loading resources).

                        • 9. Re: Reimplementing ClassPools for AS
                          kabirkhan

                          Adrian,

                          If I register a ClassLoaderPolicy that has a parent loader, I end up with a BaseClassLoader with no parent loader and an underlying ClassLoaderDomain that has a ClassLoaderToLoaderAdapter as its parent. The CLTLA contains the parent loader of the ClassLoaderPolicy. I need to be able to access what the parent loader is when constructing the pools, but don't see a way to get hold of that using the Loader interface returned by ClassLoaderDomain. Trying to hack it and use CLTLA directly also does not allow me access to the underlying parent loader.

                          Is there a way to get hold of this from the ClassLoaderSystem or elsewhere?

                          • 10. Re: Reimplementing ClassPools for AS

                            I've said this before, but I'll repeat it.

                            You shouldn't be trying to use the ClassLoaderSystem api for this.

                            The information you want is available in the ClassLoading/Modules api in the form
                            of what the parent domain is.

                            If its a sub-deployment with a Module then the parent is the classloader of the
                            parent deployment. (If it has no explicit parent domain specified.)
                            In all other cases it is the getDeterminedParentDomainName()

                            This last part is kind of an implicit rule created by the deployers
                            since only they understand what a subdeployment actually is. :-)

                            Like I said before, if we need to make something more explicit in the Module api
                            or some helper class to make your life easier then we can do that.

                            The current "implicit" rule is here:
                            http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbossas/projects/jboss-deployers/trunk/deployers-impl/src/main/java/org/jboss/deployers/plugins/classloading/AbstractLevelClassLoaderSystemDeployer.java?revision=78007&view=markup

                            Although the first comment in the code is incomplete/wrong, I've corrected here.

                             if (unit.isTopLevel() || module.getParentDomainName() != null)
                             {
                             // Top level or subdeployment with a parent domain, just create the classloader
                             return classLoaderPolicyModule.registerClassLoaderPolicy(system);
                             }
                             else
                             {
                             // Subdeployment that wants a classloader
                             ClassLoader parentClassLoader = unit.getParent().getClassLoader();
                             return classLoaderPolicyModule.registerClassLoaderPolicy(system, parentClassLoader);
                            


                            • 11. Re: Reimplementing ClassPools for AS
                              kabirkhan

                              I am trying to replicate this with my tests, although the case that I am trying to do is probably not something that will happen in the real world (i.e. in AS)?

                              I am basically creating a URLClassLoader, and then registering a policy containing a scoped child domain as its child. I then cannot get hold of the parent URLClassLoader as mentioned.

                              In AS I also don't see any way to get hold of the parent classloader from the deployment units, although in AS I belive the following to be true.
                              *The default domain is fixed up to use the loader of BaseClassLoaderDomain as its parent, which in AS means the NoAnnotationURLClassLoader.
                              *Other domains will always be children of either the default domain or an intermediate domain.

                              So my test is probably a bit too ambitious?

                              • 12. Re: Reimplementing ClassPools for AS
                                kabirkhan

                                So I either get the parent domain name from the Module and get hold of the domain using ClassLoaderSystem.getDomain() OR the parent domain name from Module is null in which case I can guess that the parent is the loader of the BaseClassLoaderDomain class?

                                • 13. Re: Reimplementing ClassPools for AS
                                  kabirkhan

                                  Adrian/Ales,

                                  I've had a look at http://www.jboss.com/index.html?module=bb&op=viewtopic&t=130122, but it is not clear to me what the difference is between ModuleRequirement and PackageRequirement? Does a module represent a jar, thus meaning import everything from that jar? And is package more fine-grained, meaning only import these packages from that jar?

                                  Whatever the answer, I suppose the Module/Package difference would also apply to the capabilities?

                                  Also, what is UsesPackageRequirement?

                                  • 14. Re: Reimplementing ClassPools for AS
                                    alesj

                                     

                                    "kabir.khan@jboss.com" wrote:

                                    Does a module represent a jar, thus meaning import everything from that jar?

                                    I wouldn't call it 'import everything from that bundle'.
                                    It's more like a plain bundle dependency, where you trust that bundle to have the packages/classes you need.
                                    e.g. you know you're using Hibernate, so instead of doing every Hibernate import,
                                    you just say I need Hibernate module, as you know that the Hibernate stuff you use it's gonna be there

                                    "kabir.khan@jboss.com" wrote:
                                    And is package more fine-grained, meaning only import these packages from that jar?

                                    Yup, exactly.
                                    It's when you need more fine-grained dependencies.

                                    "kabir.khan@jboss.com" wrote:

                                    Whatever the answer, I suppose the Module/Package difference would also apply to the capabilities?

                                    Module capabilities are filtered.
                                    e.g. VFSDeploymentClassLoaderPolicyModule
                                     // We have a module capability
                                     Object version = getVersion();
                                     Capability capability = factory.createModule(getName(), version);
                                     capabilities.add(capability);
                                    
                                     // Do we determine package capabilities
                                     ClassFilter included = getIncluded();
                                     ClassFilter excluded = getExcluded();
                                     ClassFilter excludedExport = getExcludedExport();
                                     ExportAll exportAll = getExportAll();
                                     if (exportAll != null)
                                     {
                                     Set<String> exportedPackages = PackageVisitor.determineAllPackages(roots, excludedRoots, exportAll, included, excluded, excludedExport);
                                     for (String packageName : exportedPackages)
                                     {
                                     capability = factory.createPackage(packageName, version);
                                     capabilities.add(capability);
                                     }
                                     }
                                    
                                     return capabilities;
                                    


                                    Package capabilities would normally come from jboss-classloading.xml,
                                    an explicit definition -> capabilities/package/name+version ...

                                    "kabir.khan@jboss.com" wrote:

                                    Also, what is UsesPackageRequirement?

                                    It's when some export depends on some other bundle/api.
                                    A comma-separated list of package names that are used by the
                                    exported package. Note that the use of a comma in the value requires it
                                    to be enclosed in double quotes. If this exported package is chosen as an
                                    export, then the resolver must ensure that importers of this package wire
                                    to the same versions of the package in this list.
                                    
                                    ...
                                    
                                    Classes can depend on classes in other packages. For example, when they
                                    extend classes from another package, or these other classes appear in
                                    method signatures. It can therefore be said that a package uses other packages.
                                    These inter-package dependencies are modeled with the uses directive
                                    on the Export-Package header.
                                    For example, org.osgi.service.http depends on the package javax.servlet
                                    because it is used in the API. The export definition of the
                                    org.osgi.service.http must therefore contain the uses directive with the
                                    javax.servlet package as its value.
                                    

                                    So you need to import that, in order to resolve properly,
                                    as 'uses' enforces a transitive dependency on exact version.

                                    1 2 3 Previous Next