3 Replies Latest reply on Sep 5, 2012 1:49 PM by doug.j.martin

    JBoss AS 7.1.1.Final Vault HornetQ Windows/Linux

    doug.j.martin

      I know there has been a lot of discussion on this topic and a number of issues opened/closed. I'm trying to make sense of were this problem currently stands and if there are any fixes / workarounds.

       

      The following bug reports seemed to be the most relevant. It appears 5251 was Closed/Rejected but I'm still having a number of issues on 7.1.1.Final and wasn't clear on why this was closed.

       

      https://issues.jboss.org/browse/AS7-5251

      https://issues.jboss.org/browse/AS7-5274

       

      I followed the following instruction on Windows and Linux:

      https://community.jboss.org/wiki/AS7PasswordVaultOnWindows

       

      On both platforms I get the following error when I try to use the vault with netty connectors/acceptors:

       

      2012-08-30 15:03:55,880 SEVERE [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) Failure in initialisation: java.lang.IllegalStateException: Unable to create NettyAcceptor for 0.0.0.0:5445

          at org.hornetq.core.remoting.impl.netty.NettyAcceptor.start(NettyAcceptor.java:344) [hornetq-core-2.2.13.Final.jar:]

          at org.hornetq.core.remoting.server.impl.RemotingServiceImpl.start(RemotingServiceImpl.java:240) [hornetq-core-2.2.13.Final.jar:]

          at org.hornetq.core.server.impl.HornetQServerImpl.initialisePart2(HornetQServerImpl.java:1495) [hornetq-core-2.2.13.Final.jar:]

          at org.hornetq.core.server.impl.HornetQServerImpl.access$1200(HornetQServerImpl.java:138) [hornetq-core-2.2.13.Final.jar:]

          at org.hornetq.core.server.impl.HornetQServerImpl$SharedStoreLiveActivation.run(HornetQServerImpl.java:1919) [hornetq-core-2.2.13.Final.jar:]

          at org.hornetq.core.server.impl.HornetQServerImpl.start(HornetQServerImpl.java:366) [hornetq-core-2.2.13.Final.jar:]

          at org.hornetq.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:269) [hornetq-jms-2.2.13.Final.jar:]

          at org.jboss.as.messaging.jms.JMSService.start(JMSService.java:73) [jboss-as-messaging-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]

          at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]

          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_31]

          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_31]

          at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_31]

      Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect

          at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:771) [rt.jar:1.6.0_31]

          at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:38) [rt.jar:1.6.0_31]

          at java.security.KeyStore.load(KeyStore.java:1185) [rt.jar:1.6.0_31]

          at org.hornetq.core.remoting.impl.ssl.SSLSupport.loadKeystore(SSLSupport.java:147) [hornetq-core-2.2.13.Final.jar:]

          at org.hornetq.core.remoting.impl.ssl.SSLSupport.loadKeyManagers(SSLSupport.java:168) [hornetq-core-2.2.13.Final.jar:]

          at org.hornetq.core.remoting.impl.ssl.SSLSupport.createServerContext(SSLSupport.java:63) [hornetq-core-2.2.13.Final.jar:]

          at org.hornetq.core.remoting.impl.netty.NettyAcceptor.start(NettyAcceptor.java:340) [hornetq-core-2.2.13.Final.jar:]

          ... 12 more

      Caused by: java.security.UnrecoverableKeyException: Password verification failed

          at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:769) [rt.jar:1.6.0_31]

       

      It does work for datasources and web connectors on Linux but I get the following exceptions on Windows:

       

      2012-08-30 14:40:18,208 ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 57) JBAS014612: Operation ("add") failed - address: ([

          ("subsystem" => "web"),

          ("connector" => "https")

      ]): java.lang.SecurityException: JBAS013311: Security Exception

          at org.jboss.as.security.vault.RuntimeVaultReader.retrieveFromVault(RuntimeVaultReader.java:104)

          at org.jboss.as.server.RuntimeExpressionResolver.resolvePluggableExpression(RuntimeExpressionResolver.java:45)

          at org.jboss.as.controller.ExpressionResolverImpl.resolveExpressionsRecursively(ExpressionResolverImpl.java:58) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.as.controller.ExpressionResolverImpl.resolveExpressions(ExpressionResolverImpl.java:40) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.as.controller.ModelControllerImpl.resolveExpressions(ModelControllerImpl.java:448) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.as.controller.OperationContextImpl.resolveExpressions(OperationContextImpl.java:689) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.as.controller.ParallelBootOperationContext.resolveExpressions(ParallelBootOperationContext.java:283) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.as.controller.AttributeDefinition.resolveModelAttribute(AttributeDefinition.java:242) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.as.web.WebConnectorAdd.resolveExpressions(WebConnectorAdd.java:138)

          at org.jboss.as.web.WebConnectorAdd.performRuntime(WebConnectorAdd.java:116)

          at org.jboss.as.controller.AbstractAddStepHandler$1.execute(AbstractAddStepHandler.java:50) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:385) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:272) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:200) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:311) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_31]

          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_31]

          at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_31]

          at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.0.0.GA.jar:2.0.0.GA]

      Caused by: org.jboss.security.vault.SecurityVaultException: PB00027: Vault Mismatch:Shared Key does not match for vault block:XXX and attributeName:password

          at org.picketbox.plugins.vault.PicketBoxSecurityVault.retrieve(PicketBoxSecurityVault.java:364)

          at org.jboss.as.security.vault.RuntimeVaultReader.getValue(RuntimeVaultReader.java:124)

          at org.jboss.as.security.vault.RuntimeVaultReader.getValueAsString(RuntimeVaultReader.java:112)

          at org.jboss.as.security.vault.RuntimeVaultReader.retrieveFromVault(RuntimeVaultReader.java:102)

          ... 18 more

       

      Thanks in Advance.

        • 1. Re: JBoss AS 7.1.1.Final Vault HornetQ Windows/Linux
          ctomc

          Hi,

           

          can you try with nighty build of 7.2 as there ware few issues found in this area and fixed few weeks ago.

           

          --

          tomaz

          • 2. Re: JBoss AS 7.1.1.Final Vault HornetQ Windows/Linux
            doug.j.martin

            Tomaz,

             

            I pulled down the latest 7.2.0.Alpha1-SNAPSHOT build. Let me know if this wasn't the version you intended for me to try.

             

            Looks like I had the same results with the following exceptions being thrown:

             

            2012-08-31 11:41:08,933 ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 58) JBAS014612: Operation ("add") failed - address: ([

                ("subsystem" => "web"),

                ("connector" => "https")

            ]): java.lang.SecurityException: JBAS013311: Security Exception

                at org.jboss.as.security.vault.RuntimeVaultReader.retrieveFromVault(RuntimeVaultReader.java:104)

                at org.jboss.as.server.RuntimeExpressionResolver.resolvePluggableExpression(RuntimeExpressionResolver.java:45)

                at org.jboss.as.controller.ExpressionResolverImpl.resolveExpressionsRecursively(ExpressionResolverImpl.java:58) [jboss-as-controller-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at org.jboss.as.controller.ExpressionResolverImpl.resolveExpressions(ExpressionResolverImpl.java:40) [jboss-as-controller-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at org.jboss.as.controller.ModelControllerImpl.resolveExpressions(ModelControllerImpl.java:455) [jboss-as-controller-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at org.jboss.as.controller.OperationContextImpl.resolveExpressions(OperationContextImpl.java:716) [jboss-as-controller-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at org.jboss.as.controller.ParallelBootOperationContext.resolveExpressions(ParallelBootOperationContext.java:292) [jboss-as-controller-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at org.jboss.as.controller.AttributeDefinition.resolveModelAttribute(AttributeDefinition.java:249) [jboss-as-controller-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at org.jboss.as.web.WebConnectorAdd.resolveExpressions(WebConnectorAdd.java:144)

                at org.jboss.as.web.WebConnectorAdd.performRuntime(WebConnectorAdd.java:122)

                at org.jboss.as.controller.AbstractAddStepHandler$1.execute(AbstractAddStepHandler.java:50) [jboss-as-controller-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:397) [jboss-as-controller-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:284) [jboss-as-controller-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:211) [jboss-as-controller-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:313) [jboss-as-controller-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_31]

                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_31]

                at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_31]

                at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.0.0.GA.jar:2.0.0.GA]

            Caused by: org.jboss.security.vault.SecurityVaultException: PB00027: Vault Mismatch:Shared Key does not match for vault block:XXX and attributeName:password

                at org.picketbox.plugins.vault.PicketBoxSecurityVault.retrieve(PicketBoxSecurityVault.java:364)

                at org.jboss.as.security.vault.RuntimeVaultReader.getValue(RuntimeVaultReader.java:124)

                at org.jboss.as.security.vault.RuntimeVaultReader.getValueAsString(RuntimeVaultReader.java:112)

                at org.jboss.as.security.vault.RuntimeVaultReader.retrieveFromVault(RuntimeVaultReader.java:102)

                ... 18 more

             

            2012-08-31 11:41:09,511 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-3) Started Netty Acceptor version 3.2.5.Final-a96d88c 0.0.0.0:5455 for CORE protocol

            2012-08-31 11:41:09,511 SEVERE [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) Failure in initialisation: java.lang.IllegalStateException: Unable to create NettyAcceptor for 0.0.0.0:5445

                at org.hornetq.core.remoting.impl.netty.NettyAcceptor.start(NettyAcceptor.java:344) [hornetq-core-2.2.18.Final.jar:2.2.18.Final (HQ_2_2_18_FINAL, 122)]

                at org.hornetq.core.remoting.server.impl.RemotingServiceImpl.start(RemotingServiceImpl.java:240) [hornetq-core-2.2.18.Final.jar:2.2.18.Final (HQ_2_2_18_FINAL, 122)]

                at org.hornetq.core.server.impl.HornetQServerImpl.initialisePart2(HornetQServerImpl.java:1495) [hornetq-core-2.2.18.Final.jar:2.2.18.Final (HQ_2_2_18_FINAL, 122)]

                at org.hornetq.core.server.impl.HornetQServerImpl.access$1200(HornetQServerImpl.java:138) [hornetq-core-2.2.18.Final.jar:2.2.18.Final (HQ_2_2_18_FINAL, 122)]

                at org.hornetq.core.server.impl.HornetQServerImpl$SharedStoreLiveActivation.run(HornetQServerImpl.java:1919) [hornetq-core-2.2.18.Final.jar:2.2.18.Final (HQ_2_2_18_FINAL, 122)]

                at org.hornetq.core.server.impl.HornetQServerImpl.start(HornetQServerImpl.java:366) [hornetq-core-2.2.18.Final.jar:2.2.18.Final (HQ_2_2_18_FINAL, 122)]

                at org.hornetq.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:278) [hornetq-jms-2.2.18.Final.jar:2.2.18.Final (HQ_2_2_18_FINAL, 122)]

                at org.jboss.as.messaging.jms.JMSService.start(JMSService.java:73) [jboss-as-messaging-7.2.0.Alpha1-SNAPSHOT.jar:7.2.0.Alpha1-SNAPSHOT]

                at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]

                at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.2.GA.jar:1.0.2.GA]

                at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_31]

                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_31]

                at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_31]

            Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect

                at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:771) [rt.jar:1.6.0_31]

                at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:38) [rt.jar:1.6.0_31]

                at java.security.KeyStore.load(KeyStore.java:1185) [rt.jar:1.6.0_31]

                at org.hornetq.core.remoting.impl.ssl.SSLSupport.loadKeystore(SSLSupport.java:147) [hornetq-core-2.2.18.Final.jar:2.2.18.Final (HQ_2_2_18_FINAL, 122)]

                at org.hornetq.core.remoting.impl.ssl.SSLSupport.loadKeyManagers(SSLSupport.java:168) [hornetq-core-2.2.18.Final.jar:2.2.18.Final (HQ_2_2_18_FINAL, 122)]

                at org.hornetq.core.remoting.impl.ssl.SSLSupport.createServerContext(SSLSupport.java:63) [hornetq-core-2.2.18.Final.jar:2.2.18.Final (HQ_2_2_18_FINAL, 122)]

                at org.hornetq.core.remoting.impl.netty.NettyAcceptor.start(NettyAcceptor.java:340) [hornetq-core-2.2.18.Final.jar:2.2.18.Final (HQ_2_2_18_FINAL, 122)]

                ... 12 more

            Caused by: java.security.UnrecoverableKeyException: Password verification failed

                at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:769) [rt.jar:1.6.0_31]

                ... 18 more

             

            Thanks,

             

            Doug

            • 3. Re: JBoss AS 7.1.1.Final Vault HornetQ Windows/Linux
              doug.j.martin

              Following are the config snippets I'm currently testing with:

               

              <vault>

                <vault-option name="KEYSTORE_URL" value="vault.ks"/>

                <vault-option name="KEYSTORE_PASSWORD" value="MASK-XYZ"/>

                <vault-option name="KEYSTORE_ALIAS" value="vault"/>

                <vault-option name="SALT" value="12345678"/>

                <vault-option name="ITERATION_COUNT" value="50"/>

                <vault-option name="ENC_FILE_DIR" value="\vault\\"/>

              </vault>

               

              ...

               

              <netty-acceptor name="netty-ssl-acceptor" socket-binding="messaging">

                <param key="ssl-enabled" value="true"/>

                <param key="key-store-path" value="server.ks"/>

                <param key="key-store-password" value="${VAULT::XXX::password::XYZ}"/>

                <param key="trust-store-path" value="server.ts"/>

                <param key="trust-store-password" value="${VAULT::XXX::password::XYZ}"/>

              </netty-acceptor>

               

              Does anything jump out here as being incorrect?

               

              We have a security audit looming and we certainly aren't going to pass with cleartext passwords in the config file. Any advice would be greatly appreciated.

               

              Thanks,

               

              Doug