11 Replies Latest reply: Aug 4, 2009 11:13 AM by Chris Lowe RSS

Node applications not registering correctly on initial launc

Chris Lowe Apprentice

Hello,

I've seen some unexpected behaviour a few times with mod_cluster. I have 2 nodes both of which have an application deployed using all/farm. Both nodes have been shut down along with httpd. So from a standing start, I launch httpd quickly followed by the nodes and I eventually get the following output from mod_cluster-manager:

Auto Refresh
 Node JvmRoute1 (ajp://123.234.123.100:8009): Enable Contexts Disable Contexts


Balancer: mycluster,Domain: ,Flushpackets: Off,Flushwait: 10000,Ping: 10000000,Smax: 1,Ttl: 60000000,Elected: 0,Read: 0,Transferred: 0,Connected: 0,Load: 77
Node JvmRoute2 (ajp://123.234.123.101:8009): Enable Contexts Disable Contexts

Balancer: mycluster,Domain: ,Flushpackets: Off,Flushwait: 10000,Ping: 10000000,Smax: 1,Ttl: 60000000,Elected: 0,Read: 0,Transferred: 0,Connected: 0,Load: 78


The nodes have registered with mod_cluster but the applications are missing. At this point both nodes have completely initialised and I have even left a few minutes for good measure. If I then restart httpd, I then almost instantly get what I was expecting in the first place:

Auto Refresh
 Node JvmRoute1 (ajp://123.234.123.100:8009): Enable Contexts Disable Contexts


Balancer: mycluster,Domain: ,Flushpackets: Off,Flushwait: 10000,Ping: 10000000,Smax: 1,Ttl: 60000000,Elected: 0,Read: 0,Transferred: 0,Connected: 0,Load: 76
 Virtual Host 1:

Contexts:

/, Status: ENABLED Disable
Aliases:

cluster.domain.com

 Node JvmRoute2 (ajp://123.234.123.101:8009): Enable Contexts Disable Contexts


Balancer: mycluster,Domain: ,Flushpackets: Off,Flushwait: 10000,Ping: 10000000,Smax: 1,Ttl: 60000000,Elected: 0,Read: 0,Transferred: 0,Connected: 0,Load: 78
 Virtual Host 1:

Contexts:

/, Status: ENABLED Disable
Aliases:

cluster.domain.com


I'm using mod_cluster 1.0.1.GA in unicast mode.

Any ideas on what might be causing this?

Cheers,

Chris.

  • 1. Re: Node applications not registering correctly on initial l
    Chris Lowe Apprentice

    Another slight variation on this is that I gracefully shutdown my nodes and restart them. I get the same as above: mod_cluster-manager lists the nodes but not the applications.

    Without restarting httpd or the nodes, I then deploy an application to all/farm and soon the applications are listed again in mod_cluster-manager (and the application functions fine).

  • 2. Re: Node applications not registering correctly on initial l
    Jean-Frederic Clere Master

    Look to http://www.jboss.org/mod_cluster/java/properties.html#proxy
    try to set excludedContexts (the default value contains ROOT).

  • 3. Re: Node applications not registering correctly on initial l
    Chris Lowe Apprentice

    Hi Jean-Frederic,

    Hmm, that doesn't quite work:

    Node JvmRoute2 (ajp://123.234.123.101:8009): Enable Contexts Disable Contexts
    
    
    Balancer: mycluster,Domain: ,Flushpackets: Off,Flushwait: 10000,Ping: 10000000,Smax: 1,Ttl: 60000000,Elected: 0,Read: 0,Transferred: 0,Connected: 0,Load: 76
     Virtual Host 1:
    
    Contexts:
    
    /, Status: ENABLED Disable
    Aliases:
    
    cluster.domain.com
    
     Node JvmRoute1 (ajp://123.234.123.100:8009): Enable Contexts Disable Contexts
    
    
    Balancer: mycluster,Domain: ,Flushpackets: Off,Flushwait: 10000,Ping: 10000000,Smax: 1,Ttl: 60000000,Elected: 0,Read: 0,Transferred: 0,Connected: 0,Load: -1
     Virtual Host 1:
    
    Contexts:
    
    /, Status: ENABLED Disable
    Aliases:
    
    localhost


    Notice that the alias is localhost and not cluster.domain.com.

    If the excludeContexts value is really at play here, wouldn't my context have always been blocked?

    Cheers,

    Chris.


    PS - I notice in the documentation that the default excludeLists doesn't list admin-console in 5.1; I suspect it is excluded since I'm not seeing it and the docs need a tweak?


  • 4. Re: Node applications not registering correctly on initial l
    Jean-Frederic Clere Master

    "Hmm, that doesn't quite work: ", well in the output you have / in both nodes, that looks ok for me.
    I can't reproduce it which version of AS and mod_cluster are you using?

    "Notice that the alias is localhost and not cluster.domain.com." well you have probably localhost in server.xml.

    "admin-console" that is MODCLUSTER-87. (fixed)

  • 5. Re: Node applications not registering correctly on initial l
    Chris Lowe Apprentice

    Hi Jean-Frederic,

    I'm using the following:

    mod_cluster 1.0.1.GA, linux2 x64 build;
    JBoss AS 5.1.0.GA-jdk6.

    Behind mod_cluster I have 2 nodes configured as a cluster via a TCP JGroups stack.

    Yes, my server.xml does define my host name as "localhost". My application is bound to the root context via the following jboss-web config:

    <jboss-web>
    ...
     <context-root>/</context-root>
     <virtual-host>cluster.domain.com</virtual-host>
    </jboss-web>



    You're right with the excludeLists overridden to allow ROOT, mod_cluster-manager does show an application whose context is root. However, given that the Alias is now "localhost", which application is mod_cluster picking up events for - my application or ROOT.war? If I restart httpd, then aliases are listed for both cluster.domain.com and localhost, so that would suggest that ROOT.war?

    I tried this experiment: with ROOT removed from the excludeLists, I undeployed my application from the cluster and the ROOT.war application is served. If I then deploy my application, then that is served from the same URL.

    During this experiment, mod_cluster-manager reports only the localhost alias for each node. After my application is redeployed, I restarted httpd and I given Aliases for both localhost and cluster.domain.com.

    Isn't this behaviour is undesirable? What if I add another application bound to root and with a different virtual-host, which application would win? My expectation is that several applications should be able to exist, each bound to a root context but with different virtual-host names. As applications are deployed and undeployed, mod_cluster should detect events from each of the aliases and handle requests accordingly. Have I got that wrong? Or is that beyond the scope of mod_cluster?

    Although I intend to lock down the JBoss instance by removing a good number of these services (and restricting the ones I want to keep to localhost) it would be good to prevent requests ever getting through from Apache.

    Basically, I'd like mod_cluster to only proxy for those contexts and aliases listed by mod_cluster-manager. The UseAlias option sounds like what I need here, so I tried setting UseAlias to 1 as shown in the Apache HTTPD conf section of the docs, but I get the following error from Apache:

    Invalid command 'UseAlias', perhaps misspelled or defined by a module not included in the server configuration


    Even if this option was to work, the fact that certain events from my aliased application are not getting through would cause problems?

    Finally, regardless of the whether excludeLists has been overridden or not, if I bypass VirtualHosts by accessing my Apache server directly using the IP, I can get to all of the JBoss internals: jmx-admin, admin-console - everything. I've specified the "CreateBalancers 1" option in my httpd.conf, shouldn't that mean that proxying only occurs where I have explictly specified a ProxyPass directive for a VirtualHost? Or have I misunderstood that?

    Cheers,

    Chris.

  • 6. Re: Node applications not registering correctly on initial l
    Jean-Frederic Clere Master

    I have created a JIRA MODCLUSTER-89

  • 7. Re: Node applications not registering correctly on initial l
    Chris Lowe Apprentice

    Hi Jean-Frederic, ,

    That's good, it'll be great to get one fixed. Thanks for investigating this, your time is greatly appreciated!

    Can you comment on the error message I'm getting for specifying the UseAlias parameter and that all traffic is proxied if accessing the server directly by IP?

    Let me know if I can be of any assistance with testing etc.

    Cheers,

    Chris.

  • 8. Re: Node applications not registering correctly on initial l
    Jean-Frederic Clere Master

    UseAlias is working for me. Are you really using 1.0.1.GA?

    I don't understand what you do with IP, it looks like you are accessing jboss directly.

  • 9. Re: Node applications not registering correctly on initial l
    Chris Lowe Apprentice

    Hi Jean-Frederic,

    I am using 1.0.1.GA. I've elaborated on the UseAlias and direct IP access issues below.

    Cheers,

    Chris.


    UseAlias

    From the docs:

    UseAlias: UseAlias - Check that the Alias corresponds to the ServerName (See Host Name Aliases)
    0: Don't (ignore Aliases)
    1: Check it
    Default: 0 Ignore the Alias information from the nodes.


    In my httpd.conf, I have the following line to tell mod_cluster to check match Alias with my virtual hosts ServerName:

    UseAlias 1


    But when I try and start HTTPD, I get the following error:

    Invalid command 'UseAlias', perhaps misspelled or defined by a module not included in the server configuration



    Direct IP

    In my http.conf I have defined one virtual host:

    <VirtualHost *:80>
     ServerName cluster.domain.com
    
     ProxyPass / balancer://mycluster/
    </VirtualHost>
    
    CreateBalancers 1


    From the docs:
    CreateBalancers: Define how the balancer are created in the httpd VirtualHosts, this is to allow directives like:

    ProxyPass / balancer://mycluster1/
    0: Create in all VirtualHosts defined in httpd.
    1: Don't create balancers (requires at least one ProxyPass/ProxyPassMatch to define the balancer names).
    2: Create only the main server.
    Default: 2


    I was expecting the "CreateBalancers 1" directive to prevent proxying unless I specify ProxyPass explicitly for a virtual host.

    If I access my server using the host name, then requests for http://cluster.domain.com/ result in my application being served. So far so good.

    However, if I access my web server directly using the IP - http:// 234.123.234.99/ (i.e. missing HTTP host header) then the JBoss status page is served and even though I have not overridden the excludeList to allow ROOT. In fact I can access all JBoss applications - jmx-console etc.

    Is that the expected behaviour for mod_proxy? If so, how can I lock down my server to only allow traffic via the appropriate virtual hosts setting?

    Cheers,

    Chris.



  • 10. Re: Node applications not registering correctly on initial l
    Chris Lowe Apprentice

    BTW - is there a way that I can tell that I'm actually using 1.0.1.GA?

  • 11. Re: Node applications not registering correctly on initial l
    Chris Lowe Apprentice

    Hi Jean-Frederic,

    My apologies - after running an md5sum comparison I found that one of the .so objects had not been copied when I upgraded to 1.0.1.GA.

    The UseAlias directive is no longer causing an error and this prevents the issue of accessing direct IP.

    Cheers,

    Chris.