1 Reply Latest reply: Nov 21, 2011 4:16 PM by Ron Sigal RSS

Messaging server lock up issues

vidhya Baskaran Newbie

We are having issues with a JBOSS Application server running with the following versions

jboss messaging 1.4.6

jboss remoting 2.5.2

primarily being used for messaging purposes.

 

The server often locks up  and the only way to get the clients to connect again is to restart the server.

 

Looking at the stack trace ,multiple workerthreads are waiting on this following lock

 

 

- waiting on <0x1990f6eb> (a EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$ReaderLock)

java.lang.Object.wait(Native Method)

java.lang.Object.wait(Object.java:485)

EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$ReaderLock.acquire(WriterPreferenceReadWriteLock.java:163)

org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.routeInternal(MessagingPostOffice.java:2210)

org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.route(MessagingPostOffice.java:515)

org.jboss.jms.server.endpoint.ServerConnectionEndpoint.sendMessage(ServerConnectionEndpoint.java:777)

org.jboss.jms.server.endpoint.ServerSessionEndpoint.send(ServerSessionEndpoint.java:399)

org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$send$aop(SessionAdvised.java:87)

org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(SessionAdvised$send_7280680627620114891.java)

org.jboss.jms.server.container.SecurityAspect.handleSend(SecurityAspect.java:157)

sun.reflect.GeneratedMethodAccessor233.invoke(Unknown Source)

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

java.lang.reflect.Method.invoke(Method.java:597)

org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:122)

org.jboss.jms.server.endpoint.advised.SessionAdvised$send_7280680627620114891.invokeNext(SessionAdvised$send_7280680627620114891.java)

org.jboss.jms.server.endpoint.advised.SessionAdvised.send(SessionAdvised.java)

org.jboss.jms.wireformat.SessionSendRequest.serverInvoke(SessionSendRequest.java:95)

org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:157)

org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:897)

org.jboss.remoting.transport.socket.ServerThread.completeInvocation(ServerThread.java:768)

- locked <0x69b3f1c0> (a org.jboss.remoting.transport.socket.ServerThread)

org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:721)

org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:548)

org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:234)

 

 

Attached is the settings from the remoting-bisocket-service.xml file

 

 

               <!--              

               <attribute name="writeTimeout" isParam="true">30000</attribute>

               -->

 

               <!-- End immutable parameters -->

              

               <attribute name="stopLeaseOnFailure" isParam="true">true</attribute>

              

               <!-- Periodicity of client pings. Server window by default is twice this figure -->                              

               <attribute name="clientLeasePeriod" isParam="true">10000</attribute>

               <attribute name="validatorPingPeriod" isParam="true">10000</attribute>

               <attribute name="validatorPingTimeout" isParam="true">5000</attribute>

 

               <attribute name="failureDisconnectTimeout" isParam="true">0</attribute>

               <attribute name="callbackErrorsAllowed">1</attribute>

               <attribute name="registerCallbackListener">false</attribute>

               <attribute name="useClientConnectionIdentity" isParam="true">true</attribute>

                     

                <attribute name="timeout" isParam="true">0</attribute>

 

               <!-- Max Number of connections in client pool. This should be significantly higher than

                    the max number of sessions/consumers you expect -->

               <attribute name="JBM_clientMaxPoolSize" isParam="true">1000</attribute>

              

               <!-- The maximum time to wait before timing out on trying to write a message to socket for delivery -->

               <attribute name="callbackTimeout">10000</attribute>

              

 

 

I am trying to find out if its a configuration issue or a code issue on our side ?

Is it possible that one particular client is having problems and that in turn causes the server to lock up ?

 

 

Please help.

  • 1. Re: Messaging server lock up issues
    Ron Sigal Master

    Hi Vidhya,

     

    Sorry for the delay.  Are you still having this problem?

     

    Can you see in the log which thread is holding the lock at <0x1990f6eb>?  Note that the lock exists in the JBossMessaging code.  While it is possible that there is some relevent interaction between Remoting and JBossMessaging, it's more likely that the issue is confined to JBossMessaging.

     

    Have you posted the question to the JBossMessaging user forum?

     

    -Ron