OK, firstly I apologise that this post is going to be quite lengthy but I'll try to kepp it as short as possible. Secondly I apologise because I'm not feeling the 'buzz'... more like a 'bumbling' around in the dark...
Including my code was just going to be too much touble for anyone to debug so I took a different tact and luckily found the same issues. Probably something with my understanding but I'd appreciate even a telling off...
1) Start from a known good! 2.1.0.Final / Java 1.6.0.b07
I ran the example 'clustered-queue'.
Output
[java] Sent message: This is text message 0
[java] Sent message: This is text message 1
[java] Sent message: This is text message 2
[java] Sent message: This is text message 3
[java] Sent message: This is text message 4
[java] Sent message: This is text message 5
[java] Sent message: This is text message 6
[java] Sent message: This is text message 7
[java] Sent message: This is text message 8
[java] Sent message: This is text message 9
[java] Got message: This is text message 0 from node 0
[java] Got message: This is text message 1 from node 1
[java] Got message: This is text message 2 from node 0
[java] Got message: This is text message 3 from node 1
[java] Got message: This is text message 4 from node 0
[java] Got message: This is text message 5 from node 1
[java] Got message: This is text message 6 from node 0
[java] Got message: This is text message 7 from node 1
[java] Got message: This is text message 8 from node 0
[java] Got message: This is text message 9 from node 1
[java] example complete
[java]
[java] #####################
[java] ### SUCCESS! ###
[java] #####################
There were a couple of insignificant errors saying Unable to validate user: HORNETQ.CLUSTER.ADMIN.USER
Probably caused by not securing the cluster with a valid cluster user and password
2) Copied the clustered-queue example and created a clustered-static example
Following the manual I firstly edited both the hornetq-jms.xml file to point to both servers (0+1) to point to the two server connectors
<configuration xmlns="urn:hornetq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">
<!--the connection factory used by the example-->
<connection-factory name="ConnectionFactory">
<connectors>
<connector-ref connector-name="netty-connector0"/>
<connector-ref connector-name="netty-connector1"/>
</connectors>
<entries>
<entry name="ConnectionFactory"/>
</entries>
</connection-factory>
<!--the queue used by the example-->
<queue name="exampleQueue">
<entry name="/queue/exampleQueue"/>
</queue>
</configuration>
netty-connector0 will point to server0 and netty-connector1 will point to server1 as my cluster.
The manual also shows how to wire up these two connectors in both the hornet-configuration.xml files.
This is the server1 hornetq-configuration.xml (below) however the only difference between them is that in server0 hornetq-configuration.xml the acceptor port is 5445 rather than 5446
<configuration xmlns="urn:hornetq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq
/schema/hornetq-configuration.xsd">
<clustered>true</clustered>
<!-- Connectors -->
<connectors>
<connector name="netty-connector1">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
<param key="host" value="localhost"/>
<param key="port" value="5446"/>
</connector>
<connector name="netty-connector0">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
<param key="host" value="localhost"/>
<param key="port" value="5445"/>
</connector>
</connectors>
<!-- Acceptors -->
<acceptors>
<acceptor name="netty-acceptor">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
<param key="host" value="0.0.0.0"/>
<param key="port" value="5446"/>
</acceptor>
</acceptors>
<!-- Clustering configuration -->
<cluster-connections>
<cluster-connection name="va-cluster">
<address>jms</address>
<retry-interval>500</retry-interval>
<use-duplicate-detection>true</use-duplicate-detection>
<forward-when-no-consumers>true</forward-when-no-consumers>
<max-hops>1</max-hops>
<connector-ref connector-name="netty-connector0" />
<connector-ref connector-name="netty-connector1" />
</cluster-connection>
</cluster-connections>
<!-- Other config -->
<security-settings>
<!--security for example queue-->
<security-setting match="jms.queue.exampleQueue">
<permission type="createDurableQueue" roles="guest"/>
<permission type="deleteDurableQueue" roles="guest"/>
<permission type="createNonDurableQueue" roles="guest"/>
<permission type="deleteNonDurableQueue" roles="guest"/>
<permission type="consume" roles="guest"/>
<permission type="send" roles="guest"/>
</security-setting>
</security-settings>
</configuration>
I run my new clustered-static example and get the following output
Build script found javac.
hit alternate
Using the following ant version from ..\..\..\tools\ant:
calling ..\..\..\tools\ant\bin\ant.bat -version
Apache Ant version 1.7.1 compiled on June 27 2008
calling ..\..\..\tools\ant\bin\ant.bat -Dhornetq.run_script=true
Buildfile: build.xml
run:
init:
[mkdir] Created dir:
C:\hornetq-2.1.0.Final\examples\jms\clustered-static\build
[mkdir] Created dir:
C:\hornetq-2.1.0.Final\examples\jms\clustered-static\build\classes
compile:
[echo]
src.example.dir=C:\hornetq-2.1.0.Final\examples\jms\clustered-static\src
[javac] Compiling 6 source files to
C:\hornetq-2.1.0.Final\examples\jms\clustered-static\build\classes
runExample:
[java] serverProps = -XX:+UseParallelGC -Xms256M -Xmx256M
-XX:+AggressiveOpts -XX:+UseFastAccessorMethods -Dcom.sun
.management.jmxremote
-Djava.util.logging.config.file=C:\hornetq-2.1.0.Final\examples\common/config/logging.properties
-Djava.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
-Djava.naming.factory.url.pkgs=org.jboss.naming:org
.jnp.interfaces
[java] 28/06/2010 10:01:31 org.hornetq.common.example.HornetQExample run
[java] INFO: hornetq.example.runServer is true
[java] 28/06/2010 10:01:31 org.hornetq.common.example.HornetQExample startServer
[java] INFO: starting server with config 'server0' logServerOutput true
[java] HornetQServer_0 err:[main] 10:01:33,046 WARNING [org.hornetq.core.deployers.impl.FileConfigurationParser] A IO wasn't
located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal
[java] HornetQServer_0 err:[main] 10:01:33,109 INFO [org.hornetq.core.server.impl.HornetQServerImpl] live server is starting..
[java] HornetQServer_0 err:[main] 10:01:33,140 INFO [org.hornetq.core.persistence.impl.journal.JournalStorageManager] Using NIO Journal
[java] HornetQServer_0 err:[main] 10:01:33,156 WARNING [org.hornetq.core.server.impl.HornetQServerImpl] Security risk! It has
been detected that the cluster admin user and password have not been
changed from the installation default. Please see the HornetQ user
guide, cluster chapter, for instructions on how to do this.
[java] HornetQServer_0 err:[Thread-0 (group:HornetQ-server-threads31447311-27173235)] 10:01:34,718 INFO
[org.hornetq.core.server.cluster.impl.BridgeImpl] Connecting bridge sf.va-cluster.1 to its destination
[java] HornetQServer_0 err:[Thread-1 (group:HornetQ-server-threads31447311-27173235)] 10:01:34,750 INFO
[org.hornetq.core.server.cluster.impl.BridgeImpl] Connecting bridge sf.va-cluster.0 to its destination
[java] HornetQServer_0 err:[main] 10:01:34,796 INFO [org.hornetq.core.remoting.impl.netty.NettyAcceptor] Started Netty
Acceptor version 3.2.0.Final-r2292 0.0.0.0:5445 for CORE protocol
[java] HornetQServer_0 err:[main] 10:01:34,796 INFO [org.hornetq.core.server.impl.HornetQServerImpl] HornetQ Server version
2.1.0.Final (marimbondo, 118) started
[java] HornetQServer_0 out:STARTED::
[java] 28/06/2010 10:01:34 org.hornetq.common.example.HornetQExample startServer
[java] INFO: starting server with config 'server1' logServerOutput true
[java] HornetQServer_0 err:[Thread-1 (group:HornetQ-server-threads31447311-27173235)] 10:01:34,859 INFO
[org.hornetq.core.server.cluster.impl.BridgeImpl] Bridge sf.va-cluster.0 is connected to its destination
[java] HornetQServer_1 err:[main] 10:01:35,890 WARNING [org.hornetq.core.deployers.impl.FileConfigurationParser] AIO wasn't
located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal
[java] HornetQServer_1 err:[main] 10:01:35,953 INFO [org.hornetq.core.server.impl.HornetQServerImpl] live server is starting..
[java] HornetQServer_1 err:[main] 10:01:35,984 INFO [org.hornetq.core.persistence.impl.journal.JournalStorageManager] Using
NIO Journal
[java] HornetQServer_1 err:[main] 10:01:36,000 WARNING [org.hornetq.core.server.impl.HornetQServerImpl] Security risk! It has
been detected that the cluster admin user and password have not been changed from the installation default.
Please see the HornetQ user guide, cluster chapter, for instructions on how to do this.
[java] HornetQServer_1 err:[Thread-0 (group:HornetQ-server-threads31447311-27173235)] 10:01:37,625 INFO
[org.hornetq.core.server.cluster.impl.BridgeImpl] Connecting bridge sf.va-cluster.1 to its destination
[java] HornetQServer_1 err:[Thread-1 (group:HornetQ-server-threads31447311-27173235)] 10:01:37,640 INFO
[org.hornetq.core.server.cluster.impl.BridgeImpl] Connecting bridge sf.va-cluster.0 to its destination
[java] HornetQServer_1 err:[main] 10:01:37,687 INFO [org.hornetq.core.remoting.impl.netty.NettyAcceptor] Started Netty
Acceptor version 3.2.0.Final-r2292 0.0.0.0:5446 for CORE protocol
[java] HornetQServer_1 err:[main] 10:01:37,703 INFO [org.hornetq.core.server.impl.HornetQServerImpl] HornetQ Server version
2.1.0.Final (marimbondo, 118) started
[java] HornetQServer_1 out:STARTED::
[java] 28/06/2010 10:01:37 org.hornetq.common.example.HornetQExample getContext
[java] INFO: using server0\client-jndi.properties for jndi
[java] HornetQServer_1 err:[Thread-1 (group:HornetQ-server-threads31447311-27173235)] 10:01:37,750 INFO
[org.hornetq.core.server.cluster.impl.BridgeImpl] Bridge sf.va-cluster.0 is connected to its destination
[java] HornetQServer_1 err:[Thread-0 (group:HornetQ-server-threads31447311-27173235)] 10:01:37,765 INFO
[org.hornetq.core.server.cluster.impl.BridgeImpl] Bridge sf.va-cluster.1 is connected to its destination
[java] HornetQServer_0 err:[Thread-0 (group:HornetQ-server-threads31447311-27173235)] 10:01:37,765 INFO
[org.hornetq.core.server.cluster.impl.BridgeImpl] Bridge sf.va-cluster.1 is connected to its destination
[java] 28/06/2010 10:01:37 org.hornetq.common.example.HornetQExample getContext
[java] INFO: using server1\client-jndi.properties for jndi
[java] HornetQServer_1 err:[Thread-2 (group:HornetQ-client-global-threads-15868406)] 10:01:38,109 SEVERE
[org.hornetq.core.server.cluster.impl.ClusterConnectionImpl] Failed to handle message
[java] HornetQServer_1 err:java.lang.IllegalStateException: Cannot
find binding for jms.queue.exampleQueue8c716cac-8237-11df-a46b-0015c507a7c8
[java] HornetQServer_1 err: at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.doConsumerCreated(ClusterConnectionImpl.java:774)
[java] HornetQServer_1 err: at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.onMessage(ClusterConnectionImpl.java:568)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:823)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:46)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:941)
[java] HornetQServer_1 err: at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)
[java] HornetQServer_1 err: at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
[java] HornetQServer_1 err: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[java] HornetQServer_1 err: at java.lang.Thread.run(Thread.java:619)
[java] HornetQServer_1 err:[Thread-2 (group:HornetQ-client-global-threads-15868406)] 10:01:38,125 SEVERE
[org.hornetq.core.server.cluster.impl.ClusterConnectionImpl] Failed to handle message
[java] HornetQServer_1 err:java.lang.IllegalStateException: Cannot
find binding for jms.queue.exampleQueue8c716cac-8237-11df-a46b-0015c507a7c8
[java] HornetQServer_1 err: at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.doConsumerCreated(ClusterConnectionImpl.java:774)
[java] HornetQServer_1 err: at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.onMessage(ClusterConnectionImpl.java:568)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:823)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:46)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:941)
[java] HornetQServer_1 err: at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)
[java] HornetQServer_1 err: at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
[java] HornetQServer_1 err: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[java] HornetQServer_1 err: at java.lang.Thread.run(Thread.java:619)
[java] Sent message: This is text message 0
[java] Sent message: This is text message 1
[java] Sent message: This is text message 2
[java] Sent message: This is text message 3
[java] Sent message: This is text message 4
[java] Sent message: This is text message 5
[java] Sent message: This is text message 6
[java] Sent message: This is text message 7
[java] Sent message: This is text message 8
[java] Sent message: This is text message 9
[java] Got message: This is text message 0 from node 0
[java] Got message: This is text message 1 from node 1
[java] Got message: This is text message 3 from node 0
[java] Got message: This is text message 4 from node 1
[java] Got message: This is text message 6 from node 0
[java] Got message: This is text message 7 from node 1
[java] Got message: This is text message 9 from node 0
[java] HornetQServer_1 err:[Thread-2 (group:HornetQ-client-global-threads-15868406)] 10:01:44,390 SEVERE
[org.hornetq.core.server.cluster.impl.ClusterConnectionImpl] Failed to handle message
[java] HornetQServer_1 err:java.lang.IllegalStateException: Cannot
find binding for jms.queue.exampleQueue8c716cac-8237-11df-a46b-0015c507a7c8
[java] HornetQServer_1 err: at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.doConsumerClosed(ClusterConnectionImpl.java:828)
[java] HornetQServer_1 err: at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.onMessage(ClusterConnectionImpl.java:574)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:823)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:46)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:941)
[java] HornetQServer_1 err: at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)
[java] HornetQServer_1 err: at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
[java] HornetQServer_1 err: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[java] HornetQServer_1 err: at java.lang.Thread.run(Thread.java:619)
[java] HornetQServer_1 err:[Thread-2 (group:HornetQ-client-global-threads-15868406)] 10:01:44,406 SEVERE
[org.hornetq.core.server.cluster.impl.ClusterConnectionImpl] Failed to handle message
[java] HornetQServer_1 err:java.lang.IllegalStateException: Cannot
find binding for jms.queue.exampleQueue8c716cac-8237-11df-a46b-0015c507a7c8
[java] HornetQServer_1 err: at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.doConsumerClosed(ClusterConnectionImpl.java:828)
[java] HornetQServer_1 err: at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.onMessage(ClusterConnectionImpl.java:574)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:823)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:46)
[java] HornetQServer_1 err: at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:941)
[java] HornetQServer_1 err: at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)
[java] HornetQServer_1 err: at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
[java] HornetQServer_1 err: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[java] HornetQServer_1 err: at java.lang.Thread.run(Thread.java:619)
[java] java.lang.NullPointerException
[java] at org.hornetq.jms.example.ClusteredStaticExample.runExample(ClusteredStaticExample.java:120)
[java] at org.hornetq.common.example.HornetQExample.run(HornetQExample.java:73)
[java] at org.hornetq.jms.example.ClusteredStaticExample.main(ClusteredStaticExample.java:36)
[java]
[java] #####################
[java] ### FAILURE! ###
[java] #####################
[java] Java Result: 1
BUILD FAILED
C:\hornetq-2.1.0.Final\examples\jms\clustered-static\build.xml:22: The
following error occurred while executing this lin
e:
C:\hornetq-2.1.0.Final\examples\common\build.xml:151: Example
org.hornetq.jms.example.ClusteredStaticExample failed
Total time: 13 seconds
These Failed to handle message appear to be my problem (I think the same thing is happening in my development environment)
Out of interest I ran the example again (without removing data or build directories), I get the same Failed to handle message error message, but sometimes I don't. I get some runs resulting is SUCCESS... however when I look at the output there are very strange results with some messages doubling up and other messages disappearing.
[java] Sent message: This is text message 0
[java] Sent message: This is text message 1
[java] Sent message: This is text message 2
[java] Sent message: This is text message 3
[java] Sent message: This is text message 4
[java] Sent message: This is text message 5
[java] Sent message: This is text message 6
[java] Sent message: This is text message 7
[java] Sent message: This is text message 8
[java] Sent message: This is text message 9
[java] Got message: This is text message 2 from node 0
[java] Got message: This is text message 2 from node 1
[java] Got message: This is text message 5 from node 0
[java] Got message: This is text message 3 from node 1
[java] Got message: This is text message 8 from node 0
[java] Got message: This is text message 5 from node 1
[java] Got message: This is text message 0 from node 0
[java] Got message: This is text message 7 from node 1
[java] Got message: This is text message 1 from node 0
[java] Got message: This is text message 9 from node 1
[java] Sent message: This is text message 0
[java] Sent message: This is text message 1
[java] Sent message: This is text message 2
[java] Sent message: This is text message 3
[java] Sent message: This is text message 4
[java] Sent message: This is text message 5
[java] Sent message: This is text message 6
[java] Sent message: This is text message 7
[java] Sent message: This is text message 8
[java] Sent message: This is text message 9
[java] Got message: This is text message 4 from node 0
[java] Got message: This is text message 2 from node 1
[java] Got message: This is text message 6 from node 0
[java] Got message: This is text message 3 from node 1
[java] Got message: This is text message 8 from node 0
[java] Got message: This is text message 6 from node 1
[java] Got message: This is text message 0 from node 0
[java] Got message: This is text message 8 from node 1
[java] Got message: This is text message 1 from node 0
[java] Got message: This is text message 9 from node 1
[java] Sent message: This is text message 0
[java] Sent message: This is text message 1
[java] Sent message: This is text message 2
[java] Sent message: This is text message 3
[java] Sent message: This is text message 4
[java] Sent message: This is text message 5
[java] Sent message: This is text message 6
[java] Sent message: This is text message 7
[java] Sent message: This is text message 8
[java] Sent message: This is text message 9
[java] Got message: This is text message 4 from node 0
[java] Got message: This is text message 2 from node 1
[java] Got message: This is text message 5 from node 0
[java] Got message: This is text message 3 from node 1
[java] Got message: This is text message 7 from node 0
[java] Got message: This is text message 5 from node 1
[java] Got message: This is text message 0 from node 0
[java] Got message: This is text message 7 from node 1
[java] Got message: This is text message 1 from node 0
[java] Got message: This is text message 9 from node 1
I have re-read the manual upteen times. I have tried all sorts of combinations and permutations.... but in the end I have to admit that I must be an idiot. I'm not understanding what is going on here or what to do to get it running with static connectors.
I attach my example and thank anyone who can take the time to set me on the path to hornetq success.