1 2 Previous Next 15 Replies Latest reply on Sep 8, 2010 11:06 AM by johnbailey

    Steps in building up the domain management architecture

    brian.stansberry

      We currently have standalone servers starting and server managers starting and bringing up servers. The actual subsystems within the servers are proceeding pretty well and look to be on track for meeting M1 goals. The domain management infrastucture needs some beefing up though to get it where we need it for M1.  For M1 I think we need:

       

      • Server Manager booting Domain Controller booting, if so configuring
      • Otherwise, Server Manager booting and discovering DC
      • Basic deployments via the DC (does not include graceful shutdown; perhaps not some of the more complex deployment plan variants)
      • Basic management operations via the DC
      • Basic deployments via the Standalone server's mgmt interface
      • Basic management operations via the Standalone server's mgmt interface
      • Above done via a remote Java interface
      • Good understanding of how above will be done via REST (not implemented, but thought through)
      • "Transaction log" of deployments and other domain updates (I expect this will be refined post-M1)
      • Filesystem based hot deploy on standalone server (unless Jason removes this from the list)

       

      We've talked about CLI on some threads, but it seems CLI is not actually an M1 requirement, so we need to be careful about putting resources on it.

       

      The immediate focus for myself is basic deployments via the Standalone server's mgmt interface as this allows folks working on subsystems and deployers to test properly what they are doing.

       

      Fanning out from where we are now, we need:

       

      Basic Deployment Infrastructure

       

      1. InVM programatic deployment. Now being worked by myself and John Bailey. Faciliates testing and serves as basis for
      2. Remote Java deployment interface
        1. also needs some standalone.xml configuration for the server management system's remote endpoint
      3. Filesystem based hot deploy on standalone server

       

      Domain Controller Infrastructure

       

      We  need folks to start looking into this. Once we get far enough with the  basic deployment infrastructure, I'm going to shift to it, but help from others is important.

       

      1. Domain controller configuration. What subsystems are needed for  the DC, how is their configuration expressed in domain.xml/host.xml
      2. Server Manager boots DC if host.xml says there is a local DC.
      3. Discovery mechanisms for the SM to find the DC. At least basic ones; we can add more post M1.
      4. Communication protocols between DC and SM and on to Servers

       

      General Management Update Infrastructure

       

      Emanuel Muckenhuber is looking into this. See for example https://community.jboss.org/thread/155860?tstart=0. This should also build off of and influence the basic deployment infrastructure, since deployments are just a form of management update.

       

      We need to sort out an ordered approach to how this is going to be done and divide up subpieces so others can eventually help. E.g. will we start with standalone and then expand to DomainController-based updates? Or think it through from the DomainController side and work in? Or some mix? In general we should organize the work so progress is being made even while other pieces are incomplete; e.g. don't wait until all Domain Controller Infrastructure above is in place.

       

       

      From there we bring it all together and do deployments and management updates via the DC.

       

       

      There are already JIRAs for quite a bit of this; I'll update this post later with JIRA references. Any comments, suggestions or expressions of desire to take on particular aspects are very much appreciated.

        • 1. Re: Steps in building up the domain management architecture
          johnbailey

          I am starting to take a look at the domain controller infrastructure.  I want to make sure I am on the right track. 

          Here is what I have so far:

          • The the DC configuration resides in the host.xml file
          • The DC is either referenced as a remote DC  or is configured to run local in the SM
          • A locally configured DC needs at minimum a list of interfaces and a socket binding (or maybe just port) for communication from SMs
          • Configuration from the domain.xml file is not used to configure the DC itself

          Unknowns:

          • Does the DC need subsystems (I think it will to a limited degree)
          • 2. Re: Steps in building up the domain management architecture
            brian.stansberry

            You're on the right track. Your first point is what we're shooting for -- keep the DC configuration entirely in host.xml. If the configuration gets too hairy and refers to too many things that would normally be specified in domain.xml, we'd have to reconsider.

             

            Re: does DC need subsystems, it will for sure (at least optionally) need a web server to support the REST API. And I'm sure other runtime subsystems as well (threads, security). But that doesn't mean it needs the full configuration model for those subsystems. The configuration model/schema it exposes can be much more limited.

            • 3. Re: Steps in building up the domain management architecture
              johnbailey

              For the local DC mode, is the DC actually running in the SM process, or is it launched as a separate process?  I should know this, but my memory is not giving me a clear answer.

              • 4. Re: Steps in building up the domain management architecture
                johnbailey

                The consensus on IRC was to keep it a separate process.

                • 5. Re: Steps in building up the domain management architecture
                  johnbailey
                  I put together an example XML configuration for the locally configured DC.
                  <domain-controller>
                  <local admin-binding="dc-admin">
                  <interfaces>
                  ....
                  </interfaces>
                  <jvm>
                  ....
                  </jvm>
                  <subsystems>
                  .... <!-- Some how limit -->
                  </subsystems>
                  </local>
                  </domain-controller>

                  I put together an example XML configuration for the locally configured DC.

                   

                  <domain-controller>
                       <local admin-binding="dc-admin">
                             <interfaces>
                                  ....
                             </interfaces>
                             <jvm>
                                  ....
                             </jvm>
                             <subsystems>
                                  <!-- Some how limit what subsystems can be here -->
                             </subsystems>
                        </local>
                  </domain-controller>
                  

                   

                  I am not clear on the admin-binding stuff.  Basically it is the port config we need to allow SM communication.  So we need a good name for that, and to decide if it needs a binding or just a port.

                  • 6. Re: Steps in building up the domain management architecture
                    brian.stansberry

                    Instead of using <subsystems> and trying to limit content, I'm  thinking more we have very specific elements that exactly describe what  we support


                    <remote-management>

                       <!-- HTTP-based management capabilities -->

                       <http>

                          <connectors>

                               <http-connector interface="external" port="8888"/>

                               <https-connector interface="external" port="8889"/>

                               <ajp-connector interface="external" ports="8887"/>

                          </connectors>

                     

                          <!-- Install a REST interface -->

                          <rest/>

                     

                          <!-- Deploy the admin console web app -->

                          <admin-console/>

                     

                       </http>

                     

                       <!-- Remote java interface based on remoting -->

                       <remote>...</remote>

                     

                    </remote-management>

                     


                    When the SM starts the DC Server, it could translate that into the  normal subsystem configurations that go into a Standalone, filling in  the missing pieces with defaults.

                    • 7. Re: Steps in building up the domain management architecture
                      brian.stansberry

                      On the binding configurations, the socket-binding-group in domain.xml exists to:

                       

                      1) Allow sharing of a bunch of configurations across multiple server groups via a simple reference.

                      2) Make it easier to start multiple servers on the same machine bound to the same interface by applying a simple port offset to the socket-binding-group.

                       

                      These don't cleanly apply to the DC, since there's only one DC. Cleaner I think is to stick to having the elements that need to configure a socket expose an attribute that takes an interface name (since the same named interface may be used elsewhere in host.xml) and another attribute specifying the port.

                       

                      The downside is if users want to avoid port conflicts with regular servers on the same host, they need to pick non-standard port values (e.g. don't use 8080 for HTTP). Perhaps we should have default values for the different ports that are non-conflicting with standard values.

                      • 8. Re: Steps in building up the domain management architecture
                        jason.greene

                        Brian Stansberry wrote:

                         

                        The downside is if users want to avoid port conflicts with regular servers on the same host, they need to pick non-standard port values (e.g. don't use 8080 for HTTP). Perhaps we should have default values for the different ports that are non-conflicting with standard values.

                         

                        Yes I agree we should use a non-8080 port for the DC.

                        • 9. Re: Steps in building up the domain management architecture
                          johnbailey

                          What I have for now is this:

                           <domain-controller>
                              <domain-controller>
                                 <local interface="public" port="9999">
                                     ...
                                 </local>
                              </domain-controller>
                          

                           

                          The interface element can come from either the host level interfaces or interfaces within the <local> element.

                          • 10. Re: Steps in building up the domain management architecture
                            johnbailey

                            Maybe we need to specify a more qualified name for 'interface' and 'port'. 

                            • 11. Re: Steps in building up the domain management architecture
                              johnbailey

                              The process I have laid out for booting the domain controller is as follows:

                              1. Server manager boots and parses the host.xml
                              2. If the domain controller local config exists it tells the process manager to launch the DC
                              3. The DC boots, parses the domain, opens its interface for SM communication
                              4. The SM waits for the local DC to finish booting and connects to the DC
                              5. Once connected the SM registers itself with the DC
                              6. Once the SM is registered the DC passes the latest domain to the SM
                              7. Once the SM gets the updated domain, it launches the servers

                               

                              Does this make sense?  The question I have is whether to parse the domain in the SM and pass it to the local DC, or only have the DC parse the domain?   I am leaning to the DC parsing the domain since it is in charge of it.  The only issue is there is a little bit of a delay in the SM starting the servers.  This should be a fairly small delay, but it is a delay.  All the other (remote) SM processes will have to wait for the DC anyway, so it seems better to me to have the local case act as much like the remote case as possible. 

                              • 12. Re: Steps in building up the domain management architecture
                                brian.stansberry

                                John Bailey wrote:

                                 

                                Maybe we need to specify a more qualified name for 'interface' and 'port'. 

                                 

                                Not sure what you mean here.

                                • 13. Re: Steps in building up the domain management architecture
                                  brian.stansberry

                                  John Bailey wrote:

                                   

                                  This should be a fairly small delay, but it is a delay.  All the other (remote) SM processes will have to wait for the DC anyway, so it seems better to me to have the local case act as much like the remote case as possible. 

                                   

                                  Agreed. I'd file having the SM parse the domain.xml on the list of things to look at later if we need more boot speed; i.e. a potentially premature optimization.

                                  • 14. Re: Steps in building up the domain management architecture
                                    jason.greene

                                    BTW I'll take FS deployment when the time is right.

                                    1 2 Previous Next