I am getting the below Exception after a couple of hours of running our application using HornetQ 2.0.0.GA.
2010-03-24 20:38:50,541 ERROR [_Rmx5VFhUUGFja2FnZQ._Y29tbW9ucw._Y29tbW9ucw_flybs.au] Failed to send Notification!.
javax.jms.JMSException: Timed out waiting for response when sending packet 71
Caused by: HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 71]
... 10 more
Almost at the same time (40 seconds before to be precise), JMS server log shows connection failure warning as given below
[hornetq-failure-check-thread] 20:38:06,531 WARNING [org.hornetq.core.remoting.impl.RemotingConnectionImpl] Connection failure has been detected: Did not receive ping from /10.16.189.37:52532. It is likely the client has exited or crashed without closing its connection, or the network between the server and client has failed. The connection will now be closed. [code=3]
[hornetq-failure-check-thread] 20:38:06,535 WARNING [org.hornetq.core.server.impl.ServerSessionImpl] Client connection failed, clearing up resources for session 03ac821e-3751-11df-8392-d9442de92bd4
[hornetq-failure-check-thread] 20:38:06,535 WARNING [org.hornetq.core.server.impl.ServerSessionImpl] Cleared up resources for session 03ac821e-3751-11df-8392-d9442de92bd4
This issue looks similar to the one mentioned in below topics/JIRA tickets and as per the JIRA status, they are all fixed in earlier versions but I am getting this in latest 2.0.0.GA version
I have attached my server configuration files and run.sh for reference. And btw, my server and client are in two different zones of same physical solaris machine. So there is no network overhead as well.
Any help/advice would be greatly appreciated
It looks like you had a temporary network failure..
Maybe you should look at your network config.
also look at client reconnection:
Thanks Clebert. Appreciate your help.
We are invetsigating network failure possibility as well. However, all other applications worked just fine during this period and it is odd that only HornetQ failed.
About the client reconnection you mentioned, I am a bit confused about the following statements from doc.
The window is specified in bytes, and has a default value of 1MiB.
Setting this parameter to -1 disables any buffering and prevents any re-attachment from occurring, forcing reconnect instead. The default value for this parameter is -1.
1. What is the exact default value for confirmation-window-size? 1MiB or -1?
2. As per the doc, client reconenction happens automatically. Does that mean I don't need to chnage anything from default settings to enable this?
3. If Yes, Why didn't this client reconnection happen when I had network failure? I was using the default configuration.
4. We noticed that the Timed Out Excpetion occurs when the load on JMS server is high. Is it possible that it is a bug in HornetQ?
"4. We noticed that the Timed Out Excpetion occurs when the load on JMS server is high. Is it possible that it is a bug in HornetQ"
Are you sure you didn't have any other errors on the logs? Memory errors.. etc? Anything that would interrupt the load? On that case it could be a configuraiton issue.. or maybe how you're doing acknowledgements.
I will look into 1-3 later today (if nobody does it before me). It's midnight here at my time zone when I wrote this post.
I could not find a permanent solution However, I increased the timeout parameters in hornetq-jms.xml to avoid this situation
I have seen this issue as well (http://community.jboss.org/thread/150056) when HornetQ is under load for a while but could not reproduce this with a stand alone program