1 2 Previous Next 17 Replies Latest reply on Mar 31, 2013 8:51 AM by nadeeshaan

    JBAS010900: Could not connect to remote domain controller

    foutjo

      Any help on why I might be getting this error?

       

      Using JBoss-as-7.1.1.Final Snapshot from 3/2/2012

       

      Machine #1 - domain controller

      ip=10.100.100.12

      Windows 7 64 Bit

       

      Setup the following users:

      admin

      host1 - PW= password

      host2 - PW= password

       

      Not making any changes to the host-master.xml file I start the domain controller:

       

      \bin\domain.bat --host-config=host-master.xml -Djboss.bind.address=10.100.100.12 -Djboss.bind.address.managment=10.100.100.12

       

      *starts with now errors showing up on console

       

      Machine #2 - host controller

      ip=10.100.100.4

      Fedora 10

       

      Setup the following users:

      host1 - PW= password

       

       

      Made the following changes to the host-slave.xml file.

       

      Added name=

      <host name="host1" xmlns="urn:jboss:domain:1.1">

       

      Added server-identities=


             <server-identities>
                        <secret value="cGFzc3dvcmQ="/>
            </server-identities>

       

      *secret value is base64 for "password"

       

       

      Added security-realm to domain controller

       

          <domain-controller>

              <remote host="${jboss.domain.master.address}" port="${jboss.domain.master.port:9999}" security-realm="ManagementRealm"/>

          </domain-controller>

       

       

      Using the modified host-slave.xml file start the host1 host controller:

       

      \bin\domain.bat --host-config=host-slave.xml -Djboss.bind.address=10.100.100.4 -Djboss.domain.master.address=10.100.100.12

       

       

      I keep getting the JBAS010900:Could not connect to remote domain controller.

      Below is the log info.

       

      Any help is greatly appreciated.

       

      Thanks

       

       

      14:02:01,101 DEBUG [org.jboss.as.controller.management-operation] (Controller Boot Thread) Entered VERIFY stage; waiting for service container to settle

      14:02:01,138 INFO  [org.xnio] (MSC service thread 1-2) XNIO Version 3.0.3.GA

      14:02:01,154 INFO  [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.0.3.GA

      14:02:01,162 DEBUG [org.jboss.as.server.net] (MSC service thread 1-1) Starting NetworkInterfaceService

       

      14:02:01,166 INFO  [org.jboss.remoting] (MSC service thread 1-2) JBoss Remoting version 3.2.2.GA

      14:02:01,165 DEBUG [org.jboss.as.server.net] (MSC service thread 1-1) Criteria(InetAddressMatchInterfaceCriteria(address=expression "${jboss.bind.address.management:127.0.0.1}",resolved=/127.0.0.1)) failed to accept input

       

      14:02:01,168 DEBUG [org.jboss.as.server.net] (MSC service thread 1-1) Candidate accepted addresses are: [/127.0.0.1]

       

      14:02:01,170 DEBUG [org.jboss.as.server.net] (MSC service thread 1-1) NetworkInterfaceService matched interface binding: org.jboss.as.network.NetworkInterfaceBinding@129e5e9

       

      14:02:01,204 DEBUG [org.jboss.as.host.controller] (MSC service thread 1-4) Starting Host Controller Server Inventory

      14:02:01,213 DEBUG [org.jboss.as.domain.management] (MSC service thread 1-3) Reloading properties file '/home/foutjo/jboss-as-7.1.1.Final/domain/configuration/mgmt-users.properties'

      14:02:01,215 DEBUG [org.jboss.as.domain.management] (MSC service thread 1-3) Starting 'ManagementRealm' Security Realm Service

      14:02:06,361 DEBUG [org.jboss.as.host.controller] (Controller Boot Thread) failed to connect to 10.100.100.12:9999: java.net.ConnectException: JBAS012144: Could not connect to remote://10.100.100.12:9999. The connection timed out

              at org.jboss.as.protocol.ProtocolChannelClient.connectSync(ProtocolChannelClient.java:166) [jboss-as-protocol-7.1.1.Final-SNAPSHOT.jar:7.1.1.Final-SNAPSHOT]

              at org.jboss.as.host.controller.RemoteDomainConnection.connectSync(RemoteDomainConnection.java:170) [jboss-as-host-controller-7.1.1.Final-SNAPSHOT.jar:7.1.1.Final-SNAPSHOT]

              at org.jboss.as.host.controller.RemoteDomainConnection.connect(RemoteDomainConnection.java:100) [jboss-as-host-controller-7.1.1.Final-SNAPSHOT.jar:7.1.1.Final-SNAPSHOT]

              at org.jboss.as.host.controller.RemoteDomainConnectionService.register(RemoteDomainConnectionService.java:170) [jboss-as-host-controller-7.1.1.Final-SNAPSHOT.jar:7.1.1.Final-SNAPSHOT]

              at org.jboss.as.host.controller.DomainModelControllerService.boot(DomainModelControllerService.java:343) [jboss-as-host-controller-7.1.1.Final-SNAPSHOT.jar:7.1.1.Final-SNAPSHOT]

              at org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:155) [jboss-as-controller-7.1.1.Final-SNAPSHOT.jar:7.1.1.Final-SNAPSHOT]

              at java.lang.Thread.run(Thread.java:619) [rt.jar:1.6.0_14]

        • 1. Re: JBAS010900: Could not connect to remote domain controller
          dlofthouse

          From the error I would first suspect either the AS process that is being connected to has hung (can quickly test that with the CLI on that machine) - or a firewall issue preventing the connection from completing.

           

          Can you capture a network trace between the two using something like Wireshark so we can see what is actually happening at the time of the connection?

          • 2. Re: JBAS010900: Could not connect to remote domain controller
            foutjo

            Hi Darran thanks for the help.

             

            Not familiar with the CLI command to test if host domain is hanging but I can bring up the host domain's console from the browser fine. 

             

            Not a network person so I am not familiar with Wireshark.  Is it free?  Easy to setup and run?

             

            Not sure if this helps but I was able to have these two machines play nice (communicate) when I ran them as individual servers that were part of a standalone cluster.

             

            I would really like to get this Host Domain working so that I could demo a cool way to control a cluster of servers on multiple hosts.  It looked easy based on the video

            that I viewed from the Red Hat website.  I wish the logging from this error gave some helpful info but it does not.

             

            Please let me know if there is anything else that you think I might try to get this working.

             

            Thanks again for your help.

            • 3. Re: JBAS010900: Could not connect to remote domain controller
              foutjo

              Here is some additional logging info that I was able to push out to the console.

               

              Does the additional log info give me a better clue on what might be going wrong?

               

              address.management:127.0.0.1}",resolved=/127.0.0.1) accepted input, provided bind address: /127.0.0.1

              [Host Controller] 06:15:29,781 DEBUG [org.jboss.as.server.net] (MSC service thread 1-2) Candidate accepted addresses are: [/127.0.0.1]

              [Host Controller]

              [Host Controller] 06:15:29,782 TRACE [org.jboss.as.server.net] (MSC service thread 1-2) Determining unique address from among: [/127.0.0.1]

              [Host Controller]

              [Host Controller] 06:15:29,783 TRACE [org.jboss.as.server.net] (MSC service thread 1-2) Criteria provided bind address: /127.0.0.1

              [Host Controller]

              [Host Controller] 06:15:29,783 DEBUG [org.jboss.as.server.net] (MSC service thread 1-2) NetworkInterfaceService matched interface binding: org.jboss.as.net work.NetworkInterfaceBinding@129e5e9

              [Host Controller]

              [Host Controller] 06:15:29,804 TRACE [org.jboss.remoting.endpoint] (MSC service thread 1-1) Completed open of endpoint "host1:MANAGEMENT" <1dafb4e>

              [Host Controller] 06:15:29,805 TRACE [org.jboss.remoting.endpoint] (MSC service thread 1-1) Allocated tick to 1 of endpoint "host1:MANAGEMENT" <1dafb4e> (o pened Connection provider for remote)

              [Host Controller] 06:15:29,814 TRACE [org.jboss.remoting.endpoint] (MSC service thread 1-1) Adding connection provider registration named 'remote': Remotin g remote connection provider 77eb97 for endpoint "host1:MANAGEMENT" <1dafb4e>

              [Host Controller] 06:15:29,816 DEBUG [org.jboss.as.host.controller] (MSC service thread 1-4) Starting Host Controller Server Inventory

              [Host Controller] 06:15:29,825 DEBUG [org.jboss.as.domain.management] (MSC service thread 1-2) Reloading properties file '/home/foutjo/jboss-as-7.1.1.Final /domain/configuration/mgmt-users.properties'

              [Host Controller] 06:15:29,827 DEBUG [org.jboss.as.domain.management] (MSC service thread 1-2) Starting 'ManagementRealm' Security Realm Service

              [Host Controller] 06:15:29,846 TRACE [org.jboss.remoting.endpoint] (Controller Boot Thread) Allocated tick to 2 of endpoint "host1:MANAGEMENT" <1dafb4e> (o pened Connection to /10.100.100.12:9999)

              [Host Controller] 06:15:29,858 TRACE [org.jboss.remoting.remote] (Controller Boot Thread) Attempting to connect to "/10.100.100.12:9999" with options {org. xnio.Options.SSL_ENABLED=>true,org.xnio.Options.SASL_POLICY_NOANONYMOUS=>false,org.xnio.Options.SASL_POLICY_NOPLAINTEXT=>false,org.xnio.Options.SASL_DISALL OWED_MECHANISMS=>[JBOSS-LOCAL-USER],org.xnio.Options.SASL_PROPERTIES=>[org.xnio.Property@4bb25ec],org.xnio.Options.SSL_STARTTLS=>true}

              [Host Controller] 06:15:34,989 DEBUG [org.jboss.as.host.controller] (Controller Boot Thread) failed to connect to 10.100.100.12:9999: java.net.ConnectExcep tion: JBAS012144: Could not connect to remote://10.100.100.12:9999. The connection timed out

              [Host Controller]       at org.jboss.as.protocol.ProtocolChannelClient.connectSync(ProtocolChannelClient.java:166) [jboss-as-protocol-7.1.1.Final-SNAPSHOT. jar:7.1.1.Final-SNAPSHOT]

              [Host Controller]       at org.jboss.as.host.controller.RemoteDomainConnection.connectSync(RemoteDomainConnection.java:170) [jboss-as-host-controller-7.1.1 .Final-SNAPSHOT.jar:7.1.1.Final-SNAPSHOT]

              [Host Controller]       at org.jboss.as.host.controller.RemoteDomainConnection.connect(RemoteDomainConnection.java:100) [jboss-as-host-controller-7.1.1.Fin al-SNAPSHOT.jar:7.1.1.Final-SNAPSHOT]

              [Host Controller]       at org.jboss.as.host.controller.RemoteDomainConnectionService.register(RemoteDomainConnectionService.java:170) [jboss-as-host-contr oller-7.1.1.Final-SNAPSHOT.jar:7.1.1.Final-SNAPSHOT]

              [Host Controller]       at org.jboss.as.host.controller.DomainModelControllerService.boot(DomainModelControllerService.java:343) [jboss-as-host-controller- 7.1.1.Final-SNAPSHOT.jar:7.1.1.Final-SNAPSHOT]

              [Host Controller]       at org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:155) [jboss-as-controller-7.1.1.Final-SNA PSHOT.jar:7.1.1.Final-SNAPSHOT]

              [Host Controller]       at java.lang.Thread.run(Thread.java:619) [rt.jar:1.6.0_14]

              [Host Controller]

              • 4. Re: JBAS010900: Could not connect to remote domain controller
                dlofthouse

                On the machine running the master host controller just run; -

                 

                  ./jboss-cli.sh or jboss-cli.bat

                 

                Then just enter the command: -

                 

                  connect

                 

                This will connect to the exact same port that the slave host controller is trying to connect to.

                • 5. Re: JBAS010900: Could not connect to remote domain controller
                  foutjo

                  It does fail to connect.

                  Here is the output.

                   

                  Looks like the controller is not available on localhost:9999

                  I believe mine is defined as 10.100.100.12:9999

                  Does this make a difference?

                   

                  Thanks.

                   

                  C:\apps\jboss-as-7.1.1.Final\bin>jboss-cli.bat --connect

                  WARNING JBOSS_HOME may be pointing to a different installation - unpredictable r

                  esults may occur.

                  org.jboss.as.cli.CliInitializationException: Failed to connect to the controller

                   

                          at org.jboss.as.cli.impl.CliLauncher.initCommandContext(CliLauncher.java

                  :211)

                          at org.jboss.as.cli.impl.CliLauncher.main(CliLauncher.java:190)

                          at org.jboss.as.cli.CommandLineMain.main(CommandLineMain.java:34)

                          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.

                  java:39)

                          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces

                  sorImpl.java:25)

                          at java.lang.reflect.Method.invoke(Method.java:597)

                          at org.jboss.modules.Module.run(Module.java:260)

                          at org.jboss.modules.Main.main(Main.java:291)

                  Caused by: org.jboss.as.cli.CommandLineException: The controller is not availabl

                  e at localhost:9999

                          at org.jboss.as.cli.impl.CommandContextImpl.connectController(CommandCon

                  textImpl.java:630)

                          at org.jboss.as.cli.impl.CommandContextImpl.connectController(CommandCon

                  textImpl.java:604)

                          at org.jboss.as.cli.impl.CliLauncher.initCommandContext(CliLauncher.java

                  :209)

                  • 6. Re: JBAS010900: Could not connect to remote domain controller
                    dlofthouse

                    Can you try also adding --controller=10.100.100.12:9999 to the cli command to override the address.

                     

                    One other point is you seem to be getting a warning regarding the JBOSS_HOME variable, I would suggest double checking that the variable is not set to an invalid installation on either machine otherwise there could be a version mis-match between the two.

                    • 7. Re: JBAS010900: Could not connect to remote domain controller
                      foutjo

                      My JBOSS_HOME variable is set correctly to JBoss-as-7.1.1.Final.

                       

                      When adding the --controller it seems to connect fine.

                       

                      C:\apps\jboss-as-7.1.1.Final\bin>jboss-cli.bat --connect --controller=10.100.100

                      .12:9999

                      WARNING JBOSS_HOME may be pointing to a different installation - unpredictable r

                      esults may occur.

                      [domain@10.100.100.12:9999 /]

                      • 8. Re: JBAS010900: Could not connect to remote domain controller
                        foutjo

                        Running the same CLI command on the host machine logs the following errors:

                         

                        # ./jboss-cli.sh --connect --controller=10.100.100.12:9999

                        org.jboss.as.cli.CliInitializationException: Failed to connect to the controller

                                at org.jboss.as.cli.impl.CliLauncher.initCommandContext(CliLauncher.java:211)

                                at org.jboss.as.cli.impl.CliLauncher.main(CliLauncher.java:190)

                                at org.jboss.as.cli.CommandLineMain.main(CommandLineMain.java:34)

                                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

                                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

                                at java.lang.reflect.Method.invoke(Method.java:597)

                                at org.jboss.modules.Module.run(Module.java:260)

                                at org.jboss.modules.Main.main(Main.java:291)

                        Caused by: org.jboss.as.cli.CommandLineException: The controller is not available at 10.100.100.12:9999

                                at org.jboss.as.cli.impl.CommandContextImpl.connectController(CommandContextImpl.java:630)

                                at org.jboss.as.cli.impl.CommandContextImpl.connectController(CommandContextImpl.java:604)

                                at org.jboss.as.cli.impl.CliLauncher.initCommandContext(CliLauncher.java:209)

                                ... 8 more

                        • 9. Re: JBAS010900: Could not connect to remote domain controller
                          foutjo

                          Darrin,

                           

                          Is there anything else that I can try to see what might be my problem with this one?

                           

                          You mentioned Wireshark.  Not being a network person do you think that it would be to complex for me to learn?

                           

                          Thanks.

                          • 10. Re: JBAS010900: Could not connect to remote domain controller
                            foutjo

                            Installed Wireshark and was able to capture the network traffic when the host was trying to connect to the domain controller.

                             

                            I have no clue on what I might be looking for.

                             

                            Any network experts that could give me a hand?

                             

                            Thanks.

                            • 11. Re: JBAS010900: Could not connect to remote domain controller
                              ralfbn

                              Joseph,

                               

                              I also came across this issue today. For me, being on a Mac OS X 10.7.4, it was related to the JDK: I used 7u4 for the command line. After changing back to 6u31 everything works.

                               

                              HTH, Ralf.

                              • 12. Re: JBAS010900: Could not connect to remote domain controller
                                jaikiran

                                Ralf B wrote:

                                 

                                 

                                I also came across this issue today. For me, being on a Mac OS X 10.7.4, it was related to the JDK: I used 7u4 for the command line. After changing back to 6u31 everything works.

                                 

                                So you ran into issues if the CLI used Java 7 runtime and the AS7 server was running with Java 6? Does it work if both AS7 and CLI run with Java 7 runtime? I'm trying to understand why the Java version of CLI would matter here.

                                • 13. Re: JBAS010900: Could not connect to remote domain controller
                                  ralfbn

                                  jaikiran pai wrote:

                                   

                                  Ralf B wrote:

                                   

                                   

                                  I also came across this issue today. For me, being on a Mac OS X 10.7.4, it was related to the JDK: I used 7u4 for the command line. After changing back to 6u31 everything works.

                                   

                                  So you ran into issues if the CLI used Java 7 runtime and the AS7 server was running with Java 6? Does it work if both AS7 and CLI run with Java 7 runtime? I'm trying to understand why the Java version of CLI would matter here.

                                  To explain my issue:

                                  I'm working on Java EE projects using IntellIJ 11.1.2 and JBoss 7.1.1. Things worked, but day-to-day IntellIJ stopped talking to JBoss "Server is not connected".

                                  Maven and its plugins could also not connect to JBoss on port 9999. I did not setup a cluster, but came across this posts Googling around for the same error messages.

                                   

                                  And yes, I changed things... set Java 7 as primary JDK in "Java Preferences" (OS X), but IntelliJ itself uses Java 6 even when I make Java 7 the default.

                                   

                                  When I used Java 7 for CLI:

                                  * No errors in JBoss' log files

                                  * Port 9999 was bound and accepting connections (verified with netstat and telnet)

                                  * Trying to connect to port 9999, got exception "org.jboss.as.cli.CommandLineException: The controller is not available at ..." and "Could not connect to remote://...:9999" error.

                                  * No connection through jboss-cli.sh possible

                                  * IntelliJ could start JBoss, but claimed "Server is not connected". IntelliJ spawns a process and so uses Java 7 to start JBoss.

                                   

                                  Using Java 6 for CLI:

                                  * No errors in JBoss' log files

                                  * Port 9999 was bound and accepting connections (verified with netstat and telnet)

                                  * Connecting through jboss-cli works

                                  * IntelliJ can talk to JBoss

                                   

                                  It seems that using Java 6 for all components, everything works, but using Java 7 seems to break network. A test with IntelliJ 11.1.2 (running Java 6) and GlassFish 3.1.2 works with Java 6 and 7.

                                   

                                  Please see IntelliJ http://youtrack.jetbrains.com/issue/IDEA-82687, see comments, they talk about JBoss startup/scripts and JAVA_OPTS variable.

                                  • 14. Re: JBAS010900: Could not connect to remote domain controller
                                    ctomc

                                    Hi,

                                     

                                    this looks like jdk7 for mac issue.

                                    looks like http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7159361

                                    in release notes workaround is mentionend, can you try running with:

                                    -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.KQueueSelectorProvider

                                     

                                     

                                    --

                                    tomaz

                                    1 2 Previous Next