1 2 Previous Next 19 Replies Latest reply on Aug 9, 2010 12:47 PM by dmlloyd

    Deployment APIs

    alrubinger

      I'm putting together generic Deployment APIs both for AS7 and also AS6/EAP5.  These will have 2 interfaces:

       

      1. Java
      2. REST

       

      We've talked a lot about this over the years, and this time I want to be sure all invested parties have clearly communicated their use cases and requirements.

       

      In all likelihood, the REST component will be a protocol we design which merely delegates to the underlying Java API.

       

      I'll seed the discussion with input from Max and the Tooling house.

       

      Tooling Requests

       

      1) no binary dependencies to a changing runtime API such as Profile service (suggestion: provide HTTP REST style API for it or something invokable via pure JDK JMX API)
      To be clarified.
      AS team reserves the right to change the REST API exposed between major AS versions.  If we require some common bridge / indirection to act as a delegate from the legacy view (ie the one we put in AS6) vs the real REST API for all managed operations in AS7 (of which deployment will be one aspect), that's fine too.  But we cannot lock the AS7 REST API at this time under this timeframe; it must fit into a larger picture which is still taking shape.
      On the Java API side of things, it's likely we can deliver a common view (this side *will* have binary dependencies upon a small API package).
      2) Provide a way to deploy via full archives (i.e. here is myapp.ear all zipped up please deploy)
      No problem.  To be formalized as a requirement.
      3) Provide a way to update archives "incrementally" (i.e. here is /css/style.css and /WEB-INF/classes/org/whatever/BrokenCrap.class, please put it into mywar.war note when I say archives that is just the term I use for identifiying the "root" elements.

       

      Not currently implemented / accounted for.  To be formalized as a requirement.
      4) provide API to get error information asynch.
      I deploy myapp.war when the bytes have been serialized i assume that request is done and I should be able to query later on if there were any errors doing the actual deployment.
      is also relevant when starting up the IDE and users want to know if the archives it deployed some hours ago are still looking good...
      To be clarified.
      At which level does the operation become async?  Before bytes are serialized?  Is this the responsibility of the Java API layer, or the client?  If it's the client async-ing things, we don't need to do anything specifically async in the deployment API.  If we need to know information like "Sent X bytes of Y to the server" or "Archive sent; deploying", etc...these whitebox stats must come from inside the deployment API.  In effect I'd then recommend two Java API mechanisms for deployment: one synchronous for simplicity, one async for detail and the client can block at whatever granularity it deems appropriate.
      5) query for already deployed "archives"
      Not currently implemented.  To be formalized as a requirement.
      This will amount to returning a hook / UUID of some type back to the client so it can query upon the deployment (or undeploy it).
      6) archives can also be single files (i.e. -ds.xml files)

      Not currently implemented.  To be formalized as a requirement.

       

      Right now the Containers SPI of Arquillian, from which the deployment API is to be extracted, assumes a ShrinkWrap/Archive structure for all deployables.

       

      ALR Questions

       

      • Should we support deployment of N deployables in a single call?
        • If so, must this be atomic?
        • If so, how to provide status/error reporting per deployable?
        • 1. Re: Deployment APIs
          aslak

          From Arquillian:

           

          A Query API for querying what was deployed based on a deployment unit.

           

          Request:

          • GET /content/my.ear

           

          Result:

          • modules
            • some ejb
              • stateless Service bound to java://someName impl by ServiceImpl
              • messagedriven MyListener listen to queue/MyQ
            • some war
              • servlet1 mapped to /myapp/1
              • servlet2 mapped to /myapp/2
            • etc
          • 2. Re: Deployment APIs
            pmuir

            Andrew Rubinger wrote:

             

            On the Java API side of things, it's likely we can deliver a common view (this side *will* have binary dependencies upon a small API package).

            Whats critical here is that we enforce (via testing) *binary* backwards compatibility. IOW I should be able to take a client which uses the deployment client library for JBoss AS 6, and deploy to JBoss AS 7. Without this, we are making tools jump through hoops, which makes it much less likely a good set will spring up.

             

            I would argue that any client deployment Java API we provide should, for this reason, be a thin layer over the REST API. Assuming this API is kept backwards compatible at the API level, a client build for AS6 would then work for AS7, without us having to worry about the binary linkage.

             

            Andrew Rubinger wrote

            4) provide API to get error information asynch.
            I deploy myapp.war when the bytes have been serialized i assume that request is done and I should be able to query later on if there were any errors doing the actual deployment.
            is also relevant when starting up the IDE and users want to know if the archives it deployed some hours ago are still looking good...
            To be clarified.
            At which level does the operation become async?  Before bytes are serialized?  Is this the responsibility of the Java API layer, or the client?  If it's the client async-ing things, we don't need to do anything specifically async in the deployment API.  If we need to know information like "Sent X bytes of Y to the server" or "Archive sent; deploying", etc...these whitebox stats must come from inside the deployment API.  In effect I'd then recommend two Java API mechanisms for deployment: one synchronous for simplicity, one async for detail and the client can block at whatever granularity it deems appropriate.

            IMO as soon as the "deploy" operation returns (with some handle to the deployment), it should be possible to start querying the deployment. I think that provding an immediate return (i.e just enough to just get a handle) is powerful and can be used by an interactive tool that wants to return control to the user asap, and then report status later (e.g. JBDS, CL tools) OR by something which wants to wait until the deployment is complete (e.g. Arquillian). Implementing this wait is trivial for a client, and could easily be added to a Java client API.

             

            As for the level of information, I guess the more the server can provide the better -  certainly I would love to see a real status inside my tool.

             

            Should we support deployment of N deployables in a single call?

            • If so, must this be atomic?
            • If so, how to provide status/error reporting per deployable?

            I think this will be an important thing to provide (atomic deployment of >1 deployable unit). Whether we design the API to support multiple deployables in a single call, or a way to "lock" the server, and only allow deployments from the current client until it is unlocked, I don't know.

            • 3. Re: Deployment APIs
              alrubinger

              Pete Muir wrote:

               

              Andrew Rubinger wrote:

               

              On the Java API side of things, it's likely we can deliver a common view (this side *will* have binary dependencies upon a small API package).

              Whats critical here is that we enforce (via testing) *binary* backwards compatibility. IOW I should be able to take a client which uses the deployment client library for JBoss AS 6, and deploy to JBoss AS 7. Without this, we are making tools jump through hoops, which makes it much less likely a good set will spring up.

               

              I would argue that any client deployment Java API we provide should, for this reason, be a thin layer over the REST API. Assuming this API is kept backwards compatible at the API level, a client build for AS6 would then work for AS7, without us having to worry about the binary linkage.

              I agree that our Java API should be compatible between major AS/EAP releases (to ease our tooling efforts), and looking over the common requested operations it looks like there's enough feature parity for this to be achievable.  We can consider the API an abstraction atop VDF or the new MSC equivalent.

               

              I don't think we should be inverting the layers however; we don't yet have a design in mind to which we can commit for a RESTful interface in AS7.  Therefore I'd prefer to forge ahead with a binary compatible Java API, and bring REST as the layer atop that later.  At the moment it's set to be CoolingTower which adds its own REST API to be used by tooling, and Bob's asked for a Java client to use as a delegate.

               

              Also the notion of a Java wrapper to carry out REST calls sounds less efficient than the straight invocations or RMI that the Java impl could provide.  We could also expose a JMX view.

              • 4. Re: Deployment APIs
                alrubinger

                Aslak Knutsen wrote:

                 

                From Arquillian:

                 

                A Query API for querying what was deployed based on a deployment unit.

                 

                Request:

                • GET /content/my.ear

                 

                Result:

                • modules
                  • some ejb
                    • stateless Service bound to java://someName impl by ServiceImpl
                    • messagedriven MyListener listen to queue/MyQ
                  • some war
                    • servlet1 mapped to /myapp/1
                    • servlet2 mapped to /myapp/2
                  • etc

                IMO this is a great feature request we've seen provided by other containers.  In the case of AS it's essentially a Deployment Introspection API which returns the JBossMetaData or some other, simpler, subset associated with the DeploymentUnit.

                 

                However, this is out-of-scope for the Deployment API required by tooling.  I think it'd fit nicely into into AS7, however.  Recommend we open this as a separate issue.

                 

                S,

                ALR

                • 5. Re: Deployment APIs
                  alrubinger

                  Pete Muir wrote:

                  Andrew Rubinger wrote:

                  Should we support deployment of N deployables in a single call?

                  * If so, must this be atomic?

                  * If so, how to provide status/error reporting per deployable?

                  I think this will be an important thing to provide (atomic deployment of >1 deployable unit). Whether we design the API to support multiple deployables in a single call, or a way to "lock" the server, and only allow deployments from the current client until it is unlocked, I don't know.

                  I suspect it's important too, though I'd like to know exactly what the use case is so we don't complicate the API in inventing requirements.

                   

                  Additionally, I'd need to see a use case justifying atomic deployment of N deployables across many API calls to justify the complexity in managing server state per-client to line this up.

                   

                  We can always add these things later if we skip 'em initially too; as a general rule I want things to be as slim/minimal as possible.

                   

                  S,

                  ALR

                  • 6. Re: Deployment APIs
                    alrubinger

                    From Bob:

                     

                    CoolingTower will need to export its own REST API, so I think ideally an in-process .jar we can link and use to accomplish The Real Work would be awesome. If we have to hit the AS deployment REST API directly, that's fine too. But we welcome a deployment-client.jar. We'll wrap in Ruby.

                     

                    coolingtower_deployment.jpg

                    • 7. Re: Deployment APIs
                      alrubinger

                      Where Bob refers to "Arquillian" stuff above, that's actually this new Deployment API abstraction.  Arquillian will likely end up using the same thing, and we'll have pluggable impls for each AS target.

                       

                      Question:

                      I think we need to keep the question of "getting a hook to the Deployment API" at another level.  In this case we'd be asking it of a particular server or getting it injected from somewhere.  I'm looking to avoid a ServiceLocator-looking thing.  Are we OK with getting the Deployment API from the server in a server-specific way (ie. Getting it from EAP5 might look different from AS7)? 

                       

                      I think the only one right now who needs to answer that is Bob.  Max/JBDS will be deploying through CoolingTower, so they're set.  And Arquillian knows the underlying server impl, so it can do server-specific stuff to to the Right Thing too.

                       

                      S,

                      ALR

                      • 8. Re: Deployment APIs
                        dmlloyd

                        Andrew Rubinger wrote:

                         

                        Pete Muir wrote:

                        Andrew Rubinger wrote:

                        Should we support deployment of N deployables in a single call?

                        * If so, must this be atomic?

                        * If so, how to provide status/error reporting per deployable?

                        I think this will be an important thing to provide (atomic deployment of >1 deployable unit). Whether we design the API to support multiple deployables in a single call, or a way to "lock" the server, and only allow deployments from the current client until it is unlocked, I don't know.

                        I suspect it's important too, though I'd like to know exactly what the use case is so we don't complicate the API in inventing requirements.

                         

                        Additionally, I'd need to see a use case justifying atomic deployment of N deployables across many API calls to justify the complexity in managing server state per-client to line this up.

                         

                        We can always add these things later if we skip 'em initially too; as a general rule I want things to be as slim/minimal as possible.

                         

                        S,

                        ALR

                         

                        That one is easy.  If two JARs have classloading or service deps on each other, they have to be deployed together.  The core has support for this already, at least inasmuch as it supports anything.

                        • 9. Re: Deployment APIs
                          alrubinger

                          David Lloyd wrote:

                          That one is easy.  If two JARs have classloading or service deps on each other, they have to be deployed together.  The core has support for this already, at least inasmuch as it supports anything.

                          Done and done.

                           

                          Onto the next: can be done in a single API call by the client, or we need to extend the notion of deployment "add" and "process" phases?  My note about server state is bogus; the deployment API can keep track of this state just fine.

                           

                          S,

                          ALR

                          • 10. Re: Deployment APIs
                            dmlloyd

                            Andrew Rubinger wrote:

                             

                            David Lloyd wrote:

                            That one is easy.  If two JARs have classloading or service deps on each other, they have to be deployed together.  The core has support for this already, at least inasmuch as it supports anything.

                            Done and done.

                             

                            Onto the next: can be done in a single API call by the client, or we need to extend the notion of deployment "add" and "process" phases?  My note about server state is bogus; the deployment API can keep track of this state just fine.

                             

                            S,

                            ALR

                             

                            I don't have a direct answer, but I will say this: a builder-style API accommodates this kind of multi-step process, and also maps pretty nicely to REST.

                             

                            In terms of our internal API we'd want to be able to support not only doing multiple deployments atomically, but in fact any number of potentially unrelated updates, including deployments.

                            • 11. Re: Deployment APIs
                              thomas.diesler

                              From OSGi

                               

                              We need to be able to

                               

                              1. install a bundle from a URL location
                              2. update an alrady installed bundle
                              3. query the set of installed bundles
                              4. optionally start a bundle and deployment
                              5. start a set of bundles that have already been installed

                               

                              OSGi already has a JMX based deployment API via the FrameworkMBean which we support via jbosgi-jmx. I guess this would be available in parallel and does not affect the Deployment API discussion.

                               

                              Perhaps you could prepare a (wiki) page where the formal requirements with their associated JIRAs are listed so they don't get lost in forum posts.

                               

                              cheers

                              -thomas

                              • 12. Re: Deployment APIs
                                pmuir

                                Andrew Rubinger wrote:

                                 

                                Pete Muir wrote:

                                 

                                Andrew Rubinger wrote:

                                 

                                On the Java API side of things, it's likely we can deliver a common view (this side *will* have binary dependencies upon a small API package).

                                Whats critical here is that we enforce (via testing) *binary* backwards compatibility. IOW I should be able to take a client which uses the deployment client library for JBoss AS 6, and deploy to JBoss AS 7. Without this, we are making tools jump through hoops, which makes it much less likely a good set will spring up.

                                 

                                I would argue that any client deployment Java API we provide should, for this reason, be a thin layer over the REST API. Assuming this API is kept backwards compatible at the API level, a client build for AS6 would then work for AS7, without us having to worry about the binary linkage.

                                I agree that our Java API should be compatible between major AS/EAP releases (to ease our tooling efforts), and looking over the common requested operations it looks like there's enough feature parity for this to be achievable.  We can consider the API an abstraction atop VDF or the new MSC equivalent.

                                Just to emphasze, we need not only API compatibility but also binary compatibility.

                                 

                                If you can do this directly, then that is of course better than going over http :-)

                                • 13. Re: Deployment APIs
                                  alrubinger

                                  Thomas Diesler wrote:

                                   

                                  From OSGi

                                   

                                  We need to be able to

                                   

                                  1. install a bundle from a URL location
                                  2. update an alrady installed bundle
                                  3. query the set of installed bundles
                                  4. optionally start a bundle and deployment
                                  5. start a set of bundles that have already been installed

                                   

                                  OSGi already has a JMX based deployment API via the FrameworkMBean which we support via jbosgi-jmx. I guess this would be available in parallel and does not affect the Deployment API discussion.

                                  1) Any URL with an associated stream handler (ie. remote/http?).  Or just local only ("file://")?

                                  2-5) What are the inputs denoting a "bundle"?  URL, File, ShrinkWrap Archive, or something else?  From your FrameworkMBean link, looks like these are identified via a handle of type "long".  IMO this would be outside scope for this Deployment API, though you'd be free to write a layer atop to map from your own IDs to whatever we use to represent the real Deployment.

                                   

                                  Thomas Diesler wrote:

                                  Perhaps you could prepare a (wiki) page where the formal requirements with their associated JIRAs are listed so they don't get lost in forum posts.

                                  Absolutely; that'll be the output of this lil' discussion.  As well as some interface on GitHub for review.

                                   

                                  S,

                                  ALR

                                  • 14. Re: Deployment APIs
                                    alrubinger

                                    Do we have a requirements whereby we deploy:

                                     

                                    File A, URL B, ShrinkWrap C

                                     

                                    ...together, we need to undeploy or query anything less than the full set?

                                     

                                    I'm thinking of defining a "Deployment" as a "bunch of resources deployed together".  The client then constructs the deployment:

                                     

                                    Deployment deployment = deploymentBuilder.add(File).add(URL).add(JavaArchive).build()
                                    deployer.deploy ( deployment ) ;

                                     

                                    Then undeploy and query operations may pass in the same deployment as was initially supplied by the client, rather than some handle returned by the server.  This logically groups all deployable elements of the deployment together as a unit.

                                     

                                    S,

                                    ALR

                                    1 2 Previous Next