1 2 Previous Next 15 Replies Latest reply on Aug 3, 2010 8:32 AM by aloubyansky

    System property expansion in domain/host.xml

    dmlloyd

      For or against?  Why?

       

      I'm against it for domain.xml and ambivalent for host.xml, but that's just gut feeling, not a lot of rationale.

        • 1. Re: System property expansion in domain/host.xml
          jason.greene

          It can not be allowed in domain.xml as properties can be different per system.

          • 2. Re: System property expansion in domain/host.xml
            dmlloyd

            Jason Greene wrote:

             

            It can not be allowed in domain.xml as properties can be different per system.

             

            Right and wrong.  If you do the expansion you could get variable results every time you start up the DC.  However, if all our replaceable content were defined not as, say Strings, but as, say PropertyExpressions we could make a guarantee that the property is always expanded on the host when the server runs and no sooner.

            • 3. Re: System property expansion in domain/host.xml
              brian.stansberry

              Right. We can't expand the property on the DC. Also, we can't lose track of the fact that the configured value was a system property when clients perform DomainQueries.

               

              A DomainQuery for a configuration property that was set using a system property should return:

               

              1) the fact that the configuration value was a system property

              2) the name of the property

              3) the value of that property at the level of the model being queried (the domain's value if the property belongs to a domain level element; the server-group's value if it belongs to a server-group, the server's value if it belongs to a server.)

              4) Some marker value to indicate the value was unspecified at the relevant level

              5) If the value was set at a higher level than what was queried (e.g server-group was queried but property value was set at domain level) then the level at which it was set should be indicated. Actually, this should be indicated no matter what.

              6) If the DC knows the value is overridden at lower levels, some information about this? List of overridding server groups if the domain was queried; list of overriding servers if the server-group was queried?

               

              What's fun is representing all that if system property is just part of the configuration property:

               

              <property name="clusterName">${jboss.partition.name}-SFSBCache</property>
              

               

              or even more fun

               

              <property name="clusterName">${haha}-{$bwahahaha}-cache</property>
              
              • 4. Re: System property expansion in domain/host.xml
                brian.stansberry

                My example above got me thinking about a tangential issue -- domain configuration values that are derived from other domain configuration values:

                 

                <property name="clusterName">${jboss.partition.name}-SFSBCache</property>
                

                 

                In AS 3/4/5/6, jboss.partition.name is really a "domain configuration value" (set via -g cmdline switch) that we convert to a system property as a means to pass it around. That's not the way to do it in AS 7.

                 

                How to do that generically?

                • 5. Re: System property expansion in domain/host.xml
                  dmlloyd

                  Brian Stansberry wrote:

                   

                  Right. We can't expand the property on the DC. Also, we can't lose track of the fact that the configured value was a system property when clients perform DomainQueries.

                   

                  A DomainQuery for a configuration property that was set using a system property should return:

                   

                  1) the fact that the configuration value was a system property

                  2) the name of the property

                  3) the value of that property at the level of the model being queried (the domain's value if the property belongs to a domain level element; the server-group's value if it belongs to a server-group, the server's value if it belongs to a server.)


                  Well that gets tricky.  At a server level it's simple: when the thing in question is started or deployed, the sys prop value which is set at that time is active.  But what does it mean at a domain level?  The value when the DC was started?  Or when the operation was executed to add the value?  What about if the value was added by editing domain.xml?  The value bound into the domain (because that's what we'd have to do) could vary based on factors that cannot be predicted just by looking at the domain.xml.

                   

                  I don't think we can cleanly support property values like this at the domain level or any kind of inheritance strategy.  I think that the only level that can really be "safely" supported is the server level.  Then we have a clean contract: whatever value the server had for the prop at the time it (or the deployment in question) is started is the value used, and the domain object model does not contain any values which cannot be predicted just by looking at the domain.xml file.

                   

                  Brian Stansberry wrote:

                   

                  What's fun is representing all that if system property is just part of the configuration property:

                   

                  <property name="clusterName">${jboss.partition.name}-SFSBCache</property>
                  

                   

                  or even more fun

                   

                  <property name="clusterName">${haha}-${bwahahaha}-cache</property>
                  
                  

                  Consider how javax.el.ValueExpression (http://is.gd/cUJ4k) works.  The whole thing is one constant value expression which can be expanded when the value is used.

                  • 6. Re: System property expansion in domain/host.xml
                    dmlloyd

                    Brian Stansberry wrote:

                     

                    My example above got me thinking about a tangential issue -- domain configuration values that are derived from other domain configuration values:

                     

                    <property name="clusterName">${jboss.partition.name}-SFSBCache</property>
                    

                     

                    In AS 3/4/5/6, jboss.partition.name is really a "domain configuration value" (set via -g cmdline switch) that we convert to a system property as a means to pass it around. That's not the way to do it in AS 7.

                     

                    How to do that generically?

                     

                    Well, we should still support the property (and jboss.node.name and host name etc.) because it's just damned useful to have around.  As long as the value bound into the domain for stuff like this is some kind of value expression object which can be evaluated at the server at deploy time, we should be able to support this kind of thing without any egregious violation of principles.

                    • 7. Re: System property expansion in domain/host.xml
                      dmlloyd

                      David Lloyd wrote:

                       

                      Well, we should still support the property (and jboss.node.name and host name etc.) because it's just damned useful to have around.  As long as the value bound into the domain for stuff like this is some kind of value expression object which can be evaluated at the server at deploy time, we should be able to support this kind of thing without any egregious violation of principles.

                       

                      One amendment though.  If we do support this kind of expression, we should change one behavior from what we have today: if a property is specified without a default, and the property does not exist, we should throw an exception rather than paste in the expression string (because I have yet to see a single case where that was the right thing to do).  This way misconfiguration is caught sooner rather than later, and it's easy enough to specify a default of "".

                      • 8. Re: System property expansion in domain/host.xml
                        brian.stansberry

                        David Lloyd wrote:

                         

                        Brian Stansberry wrote:

                         

                        Right. We can't expand the property on the DC. Also, we can't lose track of the fact that the configured value was a system property when clients perform DomainQueries.

                         

                        A DomainQuery for a configuration property that was set using a system property should return:

                         

                        1) the fact that the configuration value was a system property

                        2) the name of the property

                        3) the value of that property at the level of the model being queried (the domain's value if the property belongs to a domain level element; the server-group's value if it belongs to a server-group, the server's value if it belongs to a server.)


                        Well that gets tricky.  At a server level it's simple: when the thing in question is started or deployed, the sys prop value which is set at that time is active.  But what does it mean at a domain level?  The value when the DC was started?  Or when the operation was executed to add the value?  What about if the value was added by editing domain.xml?  The value bound into the domain (because that's what we'd have to do) could vary based on factors that cannot be predicted just by looking at the domain.xml.

                         

                        I realize now my thinking on this was somewhat fuzzy. There are "properties" and "system properties". The latter are more complex as the VM itself exposes mechanisms for mutating them outside the control of the domain model. That's the core of the problems you describe.

                         

                        My 3) above makes more sense if applied to a more general domain-model-only property concept; where a property is defined in the domain.xml/host.xml or via the domain API only. For such a property an inheritance strategy could work.

                        I don't think we can cleanly support property values like this at the domain level or any kind of inheritance strategy.  I think that the only level that can really be "safely" supported is the server level.  Then we have a clean contract: whatever value the server had for the prop at the time it (or the deployment in question) is started is the value used, and the domain object model does not contain any values which cannot be predicted just by looking at the domain.xml file.

                         

                        Agreed for system properties. If we can get away without any kind of domain-model-only property concept; that's great. We should design the model to minimize the need users will have for such a thing; i.e. the same value appearing in multiple places.

                        • 9. Re: System property expansion in domain/host.xml
                          jaikiran

                          Capturing some of the IRC discussion around system property value replacement in the xml:

                           

                          (09:28:29  IST) bstansberry: it's not just validation; if we resolve the system property and then inject it we lose critical information
                          (09:28:41  IST) bstansberry: <min-size>${postgresql.min-size:10}</min-size>
                          (09:28:44  IST) emuckenhuber: well yeah, that's what i meant earlier - that we can't replace that
                          (09:29:12  IST) emuckenhuber: so the console would nee to see this as well - not a random replaced number from a host
                          (09:29:26  IST) bstansberry: if we display that in a console, the admin needs to know that the configured value was ${postgresql.min-size:10} not "15"
                          (09:29:28  IST) ALR: Kind of like our current JBossMetaData vuew
                          (09:29:29  IST) ALR: *view
                          (09:29:29  IST) maeste left the room (quit: Ping timeout: 240 seconds).
                          (09:29:34  IST) bstansberry: yeah, exactly
                          (09:29:34  IST) ALR: Which we keep, but then merge
                          (09:29:44  IST) ALR: And then on the merged view, run additional defaulting
                          (09:29:48  IST) ALR: We keep copies of each
                          (09:29:51  IST) ALR: As to not lose data
                          (09:31:10  IST) jpederse: bstansberry: yeah, this is another kettle of fish - lets defer to 7.1 - it sounds like an additional service which can do the mapping correctly
                          (09:32:01  IST) jpederse: maeste: never mind - continue
                          (10:28:06  IST) Jaikiran: I'm not an xsd expert, but I guess we could have a specific type instead of plain string so that <min-size> is of "specialStringType" 
                          (10:28:35  IST) Jaikiran: then <min-size sys-prop="postgresql.min-size">10</min-size>
                          (10:28:53  IST) Jaikiran: where sys-prop is an option attribute for specialStringType (or whatever we name it)
                          (10:29:01  IST) Jaikiran: *optional
                          (10:43:47  IST) bstansberry: Jaikiran: there's a thread on system property usage at https://community.jboss.org/thread/153231?tstart=0 -- could you paste your concept there. i don't want to lose it
                          

                           

                           

                          To summarize, instead of allowing:

                           

                          <min-size>${postgres.min.size:10}</min-size>
                          

                           

                          We could perhaps have a special type, for elements like min-size which allow property substitution:

                           

                          <min-size sys-prop="postgres.min.size">10</min-size>
                          

                           

                          So if the postgres.min.size system property is set, then it's value will be used or else it will default to 10.

                          • 10. Re: System property expansion in domain/host.xml
                            emuckenhuber

                            jaikiran pai wrote:

                             

                            To summarize, instead of allowing:

                             

                            <min-size>${postgres.min.size:10}</min-size>

                             

                            We could perhaps have a special type, for elements like min-size which allow property substitution:

                             

                            <min-size sys-prop="postgres.min.size">10</min-size>
                            

                             

                            So if the postgres.min.size system property is set, then it's value will be used or else it will default to 10.

                             

                            Something like that would be one option. However it somehow makes a complex type out of an let's say "int". Also if you think about how is this going to be managed? You have a optional sys-property field next to each property in the console?

                             

                            I might not want to argue against sys properties in general - but i'm not sure this should be handled in a generic fashion by the domain model. There are definitely cases where this is useful, but maybe it should be rather part of the actual property. Where you then have to freedom to give it any type you want.

                            • 11. Re: System property expansion in domain/host.xml
                              jaikiran

                              jaikiran pai wrote:

                               

                               

                              To summarize, instead of allowing:

                               

                              <min-size>${postgres.min.size:10}</min-size>

                               

                              We could perhaps have a special type, for elements like min-size which allow property substitution:

                               

                              <min-size sys-prop="postgres.min.size">10</min-size>
                              

                               

                              So if the postgres.min.size system property is set, then it's value will be used or else it will default to 10.

                               

                              One of the things that I hadn't known until recently[1] was that we allow (in current AS versions) for the system property string expression to contain upto 2 different system property names[2]. Not sure whether we would continue to support that in AS7, but I guess it's something to keep in mind.

                               

                               

                              [1] https://jira.jboss.org/browse/JBAS-8220

                              [2] org.jboss.util.StringPropertyReplacer:

                               

                              /**
                                  * Go through the input string and replace any occurance of ${p} with
                                  * the System.getProperty(p) value. If there is no such property p defined,
                                  * then the ${p} reference will remain unchanged.
                                  * 
                                  * If the property reference is of the form ${p:v} and there is no such property p,
                                  * then the default value v will be returned.
                                  * 
                                  * If the property reference is of the form ${p1,p2} or ${p1,p2:v} then
                                  * the primary and the secondary properties will be tried in turn, before
                                  * returning either the unchanged input, or the default value.
                                  * 
                                  * The property ${/} is replaced with System.getProperty("file.separator")
                                  * value and the property ${:} is replaced with System.getProperty("path.separator").
                                  * 
                                  * @param string - the string with possible ${} references
                                  * @return the input string with all property references replaced if any.
                                  *    If there are no valid references the input string will be returned.
                                  */
                                 public static String replaceProperties(final String string)
                              
                              • 12. Re: System property expansion in domain/host.xml
                                jaikiran

                                Emanuel Muckenhuber wrote:

                                 

                                Something like that would be one option. However it somehow makes a complex type out of an let's say "int".

                                Actually, even though it's type is "int", it's being "sourced" from something outside the xml.

                                 

                                Emanuel Muckenhuber wrote:

                                Also if you think about how is this going to be managed? You have a optional sys-property field next to each property in the console?

                                 

                                Won't be each property, but only those properties which we allow to be sourced from some system property. But yes, the additional field in the console is something that I hadn't thought about.

                                • 13. Re: System property expansion in domain/host.xml
                                  brian.stansberry

                                  IMHO the additional field(s) in the console is a good thing. The way AS 5/6 work, the user loses information with the console; if they override a value that was set via a system property they don't necessarily know they've done that and they have no way to state what their intent is:

                                   

                                  1) Permanently change the config from a system property substitution to a fixed value

                                  2) Temporarily change from the system property to a fixed value

                                  3) Change the value of the system property at the same time the config property is changed.

                                   

                                  With what you're proposing Jaikiran, the user has options:

                                   

                                  1) Change the current value and the default, but leave the config as a system property substitution, i.e.

                                   

                                   

                                  <min-size sys-prop="postgres.min.size">10</min-size>

                                  becomes
                                  <min-size sys-prop="postgres.min.size">15</min-size>
                                  

                                  This choice should only be allowed if the system property isn't actually set anywhere. Otherwise, if it's set somewhere to "12", the configuration becomes inconsistent; the config says the value should be 12 but the actual value is 15. That opens a bit of a can of worms, since the DC would need to know whether the system property is set on any affected server. If system properties are somehow coming in via the command line, the DC may not know that.

                                   

                                  2) Change the current value, leave the config as a system property subsitution with default 10, and change the system property

                                   

                                  <system-properties>

                                      <property name="postgres.min.size" value="12"/>

                                  </system-properties>

                                   

                                  becomes

                                   

                                  <system-properties>

                                      <property name="postgres.min.size" value="15"/>

                                  </system-properties>

                                   

                                  The console GUI would need to make it clear that this is what is happening.

                                   

                                  3) Change the current value, getting rid of the system property substitution:

                                   

                                  <min-size>15</min-size>
                                  • 14. Re: System property expansion in domain/host.xml
                                    dmlloyd

                                    Another, completely different idea could be to allow system properties to override values based on a coordinate system.

                                     

                                    For example, if I have:

                                     

                                       <bounded-queue-executor name="MyPool" ...>
                                           <max-threads count="10"/>
                                           ...
                                       </bounded-queue-executor>
                                    

                                     

                                    my subsystem could automatically search for an override value at "jboss.as.threads.executor.MyPool.max-threads.count=10" or some other structurally-derived property name.

                                    1 2 Previous Next