1 Reply Latest reply on Sep 27, 2011 4:45 PM by ecimon

    Distributed queue and several cluster partitions

    ecimon

      Hi,

       

      I'm running four JBoss AS 5.1 instances on a single, non-multihomed server (for testing purposes only) and I'm having some difficulties with my JBoss Messaging setup.

       

      My cluster layout is as follows:

       

      Host
      PartitonPorts
      ServerPeerID
      service0ServicePartitionports-default1
      service1ServicePartitionports-012
      web0WebPartitionports-023
      web1WebPartitionports-034

       

      I'm also using a shared PostgreSQL instance as a persistence provider inside the ServicePartion (no messaging related changes were applied to the WebPartion nodes). I used the attached example  (docs/)  to create my server/all/deploy/messaging/postgresql-persistence-service.xml on all service nodes. I also applied two minor changes to the PostOffice mbean definition: Clustered and FailoverOnNodeLeave attributes were set to true.

       

      My JMS queue is deployed as a part of the service.ear archive (which goes to service nodes only):

      {code:xml}

      <server>

         <mbean code="org.jboss.jms.server.destination.QueueService"

            name="jboss.messaging.destination:service=Queue,name=DictionaryTestQueue"

            xmbean-dd="xmdesc/Queue-xmbean.xml">

            <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends>

            <depends>jboss.messaging:service=PostOffice</depends>

            <attribute name="Clustered">true</attribute>

         </mbean>

      </server>

      {code}

       

      As long as both Web nodes are down everything works fine - I'm able to connect to queue/DictionaryTestQueue with a standalone client (using HA-JNDI) and send a simple text message, which is consequently load balanced across both service nodes.

       

      However, as soon as I launch a Web node, I ocasionally get the following exception (same, standalone client):

      {code}

      Caused by: javax.jms.JMSException: Failed to route Reference[57607127264149508]:RELIABLE to DictionaryTestQueue

                at org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendMessage(ServerConnectionEndpoint.java:757)

                at org.jboss.jms.server.endpoint.ServerSessionEndpoint.send(ServerSessionEndpoint.java:399)

                at org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$send$aop(SessionAdvised.java:87)

                at org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeTarget(SessionAdvised$send_7280680627620114891.java)

                at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)

                at org.jboss.jms.server.container.SecurityAspect.handleSend(SecurityAspect.java:157)

                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

                at java.lang.reflect.Method.invoke(Method.java:597)

                at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122)

                at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

                at org.jboss.jms.server.endpoint.advised.SessionAdvised.send(SessionAdvised.java)

                at org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:95)

                at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:143)

                at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:891)

                at org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:744)

                at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:697)

                at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:551)

                at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:232)

                at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:211)

                at org.jboss.remoting.Client.invoke(Client.java:1724)

                at org.jboss.remoting.Client.invoke(Client.java:629)

                at org.jboss.remoting.Client.invoke(Client.java:617)

                at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:189)

                at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:160)

                at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$send$aop(ClientSessionDelegate.java:499)

                at org.jboss.jms.client.delegate.ClientSessionDelegate$send_6145266547759487588.invokeTarget(ClientSessionDelegate$send_6145266547759487588.java)

                at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)

                at org.jboss.jms.client.container.SessionAspect.handleSend(SessionAspect.java:661)

                at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect_z_handleSend_194171714.invoke(SessionAspect_z_handleSend_194171714.java)

                at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

                at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:114)

                at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)

                at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

                at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)

                at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)

                at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

                at org.jboss.jms.client.delegate.ClientSessionDelegate.send(ClientSessionDelegate.java)

                at org.jboss.jms.client.container.ProducerAspect.handleSend(ProducerAspect.java:269)

                at org.jboss.aop.advice.org.jboss.jms.client.container.ProducerAspect_z_handleSend_194171714.invoke(ProducerAspect_z_handleSend_194171714.java)

                at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

                at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)

                at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:86)

                at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

                at org.jboss.jms.client.delegate.ClientProducerDelegate.send(ClientProducerDelegate.java)

                at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:164)

                at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:207)

                at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:145)

                at org.jboss.jms.client.JBossMessageProducer.send(JBossMessageProducer.java:136)

      {code}

       

      So it looks to me like a service partiion node is trying to use a web node for queue lookup. Is this the correct behaviour or am I missing something here?

       

      What would be the recommended approach here? Should I define my application queues/setup messaging on all cluster nodes (both partions) or should I keep it separate?