7 Replies Latest reply on Mar 25, 2010 4:01 PM by jeffdelong

    Using Drools Guvnor 5.1 as a service repository

    jeffdelong

      I have spent some time working with the latest version of Drools Guvnor 5.1 to see if it could be used to provide a basic repository for service related artifacts. I have created a "service repository schema" by adding a set of categories, statuses, and a GlobalServices package, and exporting this repository. You can find a copy attached (service_repository.xml)

       

      I am also imported this repository into a different instance of Guvnor, and imported most, if not all, of the artifacts from the SOA Workshop Labs as a test of this schema. I have exported this schema and attached it as well (policyquote_service_repository.xml).

       

      I have described all of this in a step-by-step manner in the attached pdf document (Service Repository with Drools Guvnor 5.1.pdf).

       

      For those of you who are interested, please review and provide comments.

       

      Jeff

        • 1. Re: Using Drools Guvnor 5.1 as a service repository
          objectiser

          Hi Jeff

           

          Good work - I'll try to find some time to try it out myself, but just a couple of comments.

           

          Some of the stored items are artifacts, while others are deployable units (jar, esb, etc). At the moment there does not appear to be any way of linking the source artifacts with the deployment unit - or is it possible to somehow identify dependencies?

           

          This also leads to the question of how we would build those components, and understand which versions of the various artifacts were used to build them. If such a build system was possible, this could automatically establish the dependencies with the generated deployment unit, making it possible to understand when changes to various artifacts would impact deployed services.

           

          Are there any ideas of how these deployment units, that are in the status 'Deployed', will relate to actual deployment in a test or production environment? And equallty how do we then define when an updated deployment unit should be deployed - i.e. is there a concept of a subsequent version of deployment unit X.esb, being in a different status (i.e. PreDeploy)?

           

          To avoid confusion, possibly a concrete example would be useful.

           

          1) We have 'PolicyQuoteService.esb' in status 'Deployed', currently running on the SOA platform. This is based on (for simplicity) a single jboss-esb.xml file, with no other artifacts. So there will be a dependency relationship between the .esb file and the jboss-esb.xml.

           

          2) We then decide that we want to update the service, so some changes are made to the jboss-esb.xml file and these added to Guvnor as a new version of the file.

           

          3) A build mechanism is then invoked that understands that the .esb file packages the jboss-esb.xml - and the result of this is a new version of PolicyQuoteService.esb, but in a status 'PreDeployed'. So one point is whether the status is associated with the file, or with the specific version of the file?

           

          4) This deployment unit could then possibly be moved to status 'Test', where it could be deployed to a test environment automatically - along with any other deployment units that it is dependent upon - or changed to status 'Deploy' to deploy in the production environment, possibly subject to suitable authorisation.

           

          Sorry, this reply has got rather involved, and I know that some of this would not be possible now - but just wondering whether this is a use-case we want to explore, and how much of it may be possible with Guvnor now.

           

          Regards

          Gary

          • 2. Re: Using Drools Guvnor 5.1 as a service repository
            bcarothers

            Jeff,  I really like how you were able to repurpose Guvnor for SOA artifacts without modifying the code.  Neat hack!

             

            objectiser wrote:

             

            Some of the stored items are artifacts, while others are deployable units (jar, esb, etc). At the moment there does not appear to be any way of linking the source artifacts with the deployment unit - or is it possible to somehow identify dependencies?

             

            I think that would be relatively straightforward with a ModeShape sequencer if ModeShape was used as the JCR implementation under Guvnor.  There is already a sequencer that can explode a .zip file (or a .jar or .esb, for that matter). 

             

            This also leads to the question of how we would build those components, and understand which versions of the various artifacts were used to build them. If such a build system was possible, this could automatically establish the dependencies with the generated deployment unit, making it possible to understand when changes to various artifacts would impact deployed services.

            Would the intent be to build and deploy from the repository, or to build on some other environment, upload the build artifact (e.g., ESB archive) to the repository, and then let the repository decompose the build artifact into its constituent artifacts for searching and analysis?

             

            Are there any ideas of how these deployment units, that are in the status 'Deployed', will relate to actual deployment in a test or production environment? And equallty how do we then define when an updated deployment unit should be deployed - i.e. is there a concept of a subsequent version of deployment unit X.esb, being in a different status (i.e. PreDeploy)?

            Great question.  Are people (internal to JBoss) expecting services to be published directly from the repository or through JON?  My supposition is that customers would not punch a hole through their firewall to allow a service repo to push deployment units to prod servers. 

             

            3) A build mechanism is then invoked that understands that the .esb file packages the jboss-esb.xml - and the result of this is a new version of PolicyQuoteService.esb, but in a status 'PreDeployed'. So one point is whether the status is associated with the file, or with the specific version of the file?

             

            I think that tracking the deployment footprint of services across environments is important and it would have to be done on a per-version basis.  For example, FooService 1.0 may be in production while FooService 1.1 is in integration testing and FooService 2.0 is being developed.

            • 3. Re: Using Drools Guvnor 5.1 as a service repository
              objectiser

              bcarothers wrote:

               

              Would the intent be to build and deploy from the repository, or to build on some other environment, upload the build artifact (e.g., ESB archive) to the repository, and then let the repository decompose the build artifact into its constituent artifacts for searching and analysis?

               

               

              Not sure - but one issue with the explode approach is whether it would successfully identify where those artifacts already exist in the repository, and setup dependencies on those existing artifacts+version.

               

              Great question.  Are people (internal to JBoss) expecting services to be published directly from the repository or through JON?  My supposition is that customers would not punch a hole through their firewall to allow a service repo to push deployment units to prod servers. 

               

              Good point - I guess we need to understand what is the relationship between Guvnor and a management system like JON needs to be. Moving a package into 'Deploy' or 'Test' state could just be used as a notification to the management system, to trigger the deployment process - whether automatically, or manually performed by an administrator.

               

              I think that tracking the deployment footprint of services across environments is important and it would have to be done on a per-version basis.  For example, FooService 1.0 may be in production while FooService 1.1 is in integration testing and FooService 2.0 is being developed.

              Yes I think so to - and one point this raises - if the Gurnor status field is going to be used for values such as 'Business Analysis', 'Service Oriented Analysis' etc - as well as potentially 'Integration Testing', 'Production' etc - then we will probably need some validation to only permit certain status values to be applied to certain artifact or package types.

              • 4. Re: Using Drools Guvnor 5.1 as a service repository
                bcarothers

                objectiser wrote:

                 

                bcarothers wrote:

                 

                Would the intent be to build and deploy from the repository, or to build on some other environment, upload the build artifact (e.g., ESB archive) to the repository, and then let the repository decompose the build artifact into its constituent artifacts for searching and analysis?

                 

                 

                Not sure - but one issue with the explode approach is whether it would successfully identify where those artifacts already exist in the repository, and setup dependencies on those existing artifacts+version.

                 

                I think that the repository could identify most of those dependencies.  It would also need some help from the metadata in the build artifact and, probably, some occasional manual input. 

                 

                As a concrete example, suppose that FooService.esb is uploaded to the repository and consists of a JAR that contains classes with JPA annotations, a JAR that contains classes with JSR-181 annotations (or JSR-311 or any other annotation to a loosely-coupled interface), and a jboss-esb.xml file.  I would expect some version of the following process to occur:

                 

                1. Explode - The repository could explode the archive and JARs into its internal storage

                2. Analyze - The repository could then parse out the annotations from the classes.  In this example, that would imply the required database structure and enable generation of exposed WSDL for the services.  The repository could also parse the jboss-esb.xml file to determine any dependencies.  Ideally, the repository would also associate (or suggest associations for) any related design artifacts from a known document repository and tie the classes to source code in a source code repository.

                3. Publish - The repository would then publish the service to its search index, allowing future services to dynamically identify dependencies on this service more easily.  (Yes, this is the hand-waving part, although I do have some ideas for how it could work).

                 

                What do you think about this process?

                Good point - I guess we need to understand what is the relationship between Guvnor and a management system like JON needs to be.

                IMHO, it would be nice if JON could pull deployment units from the repository and push them out to the live servers.  Customers seem to buy into that deployment model. 

                 

                Yes I think so to - and one point this raises - if the Gurnor status field is going to be used for values such as 'Business Analysis', 'Service Oriented Analysis' etc - as well as potentially 'Integration Testing', 'Production' etc - then we will probably need some validation to only permit certain status values to be applied to certain artifact or package types.

                 

                I agree.  I think Jervis's work on Guvnor-SOA has this concept of documentation artifact vs. deployable artifact.

                • 5. Re: Using Drools Guvnor 5.1 as a service repository
                  jeffdelong

                  Gary and Brian: these are all great comments and point out things that need to be taken into account by the developers of the "real" service repository, as well as things that we might be able to support using Guvnor 5.1 with some thought and a bit more work.

                   

                  What I have proposed, as Brian states, is "really nice hack" to provide the basic outline of a service repository. All of the dependency management is through the placement of the artifacts within the same package / service. The governance process is not enforced by Guvnor, statuses just provide a way to help the developer identify which phase of the lifecycle the artifact has reached. There is a "hole" here, how to link artifacts in the ServicesGlobal area to the service; the easiest approach would be for the developer to add their own category for each service / package they create, and add this category to each artifact in the ServicesGlobal package as appropriate. This is actually the purpose of the Guvnor "Global Area", but this is new to Guvnor 5.1 and JBDS Eclipse Guvnor Tooling has not been updated to upload artifacts to it, so I had to invent my own.

                   

                  For service versioning, since packages cannot be versioned, I would probably recommend a new package be created, and then copy relevant assets to the new version, then take the new version through the lifecycle using the status fields. But to be honest I have not thought this through or experimented with it yet.

                   

                  As far as archive creation or explosion currently I would have the developer build the deployment archive in JBDS and then update the deployment archive with the latest version. Although more capability could be added here without new coding in Guvnor. It would be possible to provide an ANT script that used the webdav interface to pull down the artifacts to a server and build the deployment artifacts, and then store them back in the repository. The same could be probably be done for artifact explosion (I think both of these would take more effort than what I have currently done).

                  • 6. Re: Using Drools Guvnor 5.1 as a service repository
                    objectiser

                    Sorry Jeff, we were getting carried away. But having something concrete like you have produced makes it easier to brainstorm ideas

                     

                    Regards

                    Gary

                    • 7. Re: Using Drools Guvnor 5.1 as a service repository
                      jeffdelong

                      No problem Gary, I really want to get people's feedback on what I have proposed so that I can think about how I might extend it to provide a bit more capabilty. In particular in my last post I responded to one of your comments with:

                       

                      "There is a "hole" here, how to link artifacts in the ServicesGlobal area to the service; the easiest approach would be for the developer to add their own category for each service / package they create, and add this category to each artifact in the ServicesGlobal package as appropriate."

                       

                      I have tested this out and this is indeed the way to go. In fact all artifacts should be tagged accordingly. For example with PolicyQuoteProcess, the policyQuoteRequest.xsd, policyQuoteResponse.xsd, policyQuoteFault.xsd, and policyQuoteBase.xsd would all be stored under the ServiceGlobal package. They would be tagged with Categories Services/Orchestrated Services/PolicyQuoteProcessService, Services/Entity Services/PolicyQuoteEntityService, Services/Task Services/PolicyQuoteCalculationService, as well as XML Schemas. Other artififacts, such as the PolicyQuoteProcess.cdm and PolicyQuoteProcess.bpel would also be tagged with Services/Orchestrated Services/PolicyQuoteProcessService. By doing this, one could open the Asset category view and see all of these assets under Services/Orchestrated Services/PolicyQuoteProcessService.

                       

                      On a related note, I have updated the Lab instructions to use Guvnor as a service repository. It makes it easier to manage the files, particuarly those that are used in multiple projects (e.g. XSDs.) I will post the lab instructions document for people to take a look at.