-
1. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
greyfairer2 May 27, 2011 11:18 AM (in response to greyfairer2)BTW, I'm using HornetQ 2.1.2.Final with Netty 3.2.1.Final (included in JBoss 6.0.0.Final)
-
2. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
clebert.suconic May 27, 2011 11:49 AM (in response to greyfairer2)I wouldn't recommend anyone using 2.1.2 due to a journal bug that was fixed on 2.2.2
-
3. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
clebert.suconic May 27, 2011 12:00 PM (in response to clebert.suconic)I'm not aware of tests being done with mod-proxy and the servlet transport. It will probably need some tweaking.
If you want to embrace that route, you can count on the forums here to help you with issues. However I don't think anyone from the HQ team (including me) would be able to spend time on testing anything until mid next week.
-
4. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
greyfairer2 May 31, 2011 11:52 AM (in response to clebert.suconic)Hi there,
I just upgraded to HornetQ 2.2.2 and Netty 3.2.4, and apache httpd 2.2.19. I could reproduce the exact same issue with netty-servlet transport via mod_proxy_ajp:
javax.jms.JMSException: Timed out waiting for response when sending packet 43
at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:276)
at org.hornetq.core.client.impl.ClientSessionImpl.commit(ClientSessionImpl.java:527)
at org.hornetq.core.client.impl.DelegatingSession.commit(DelegatingSession.java:157)
at org.hornetq.jms.client.HornetQSession.commit(HornetQSession.java:234)
...
Caused by: HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 43]
... 19 more
For the http transport there is a new HttpChunkAggregator in 2.2.2. I will try tomorrow to see if that gives better results.
-
5. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
greyfairer2 May 31, 2011 11:54 AM (in response to greyfairer2)BTW, I also stumbled upon another issue I reported to netty and patched locally: https://issues.jboss.org/browse/NETTY-407
-
6. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
greyfairer2 Jun 1, 2011 12:13 PM (in response to greyfairer2)OK, So I tried with http-enabled transport listening on port 8081, and http-enabled client connecting to apache mod_proxy_http on port 80 with "ProxyPass /hornetq http://localhost:8081/hornetq".
Connecting directly to port 8081 everything works, but on port 80, I never get a successful connection:
Caused by: javax.jms.JMSException: Timed out waiting for response when sending packet 104
at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:276)
at org.hornetq.core.client.impl.ClientSessionImpl.addMetaData(ClientSessionImpl.java:1104)
at org.hornetq.core.client.impl.DelegatingSession.addMetaData(DelegatingSession.java:567)
at org.hornetq.jms.client.HornetQConnection.addSessionMetaData(HornetQConnection.java:603)
at org.hornetq.jms.client.HornetQConnection.authorize(HornetQConnection.java:591)
at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:684)
at org.hornetq.jms.client.HornetQConnectionFactory.createQueueConnection(HornetQConnectionFactory.java:131)
at org.hornetq.jms.client.HornetQConnectionFactory.createQueueConnection(HornetQConnectionFactory.java:126)
... 5 more
Caused by: HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 104]
... 14 more
If I trace with wireshark, I see that you try to send 2 POST requests on the same connection:
=======================================================================
POST http://local.hornetq.qaweb.medical.barco.com:8081/hornetq1/HornetQServlet HTTP/1.1
Host: local.hornetq.qaweb.medical.barco.com:8081
Content-Length: 82
N$$67d4a7b6-8c65-11e0-8c8b-005056c00008
z
=======================================================================
POST http://local.hornetq.qaweb.medical.barco.com:8081/hornetq1/HornetQServlet HTTP/1.1
Host: local.hornetq.qaweb.medical.barco.com:8081
Content-Length: 21
P
=======================================================================
The Acceptor send 2 responses back:
=======================================================================
HTTP/1.1 200 OK
Content-Length: 17
z
=======================================================================
HTTP/1.1 200 OK
Content-Length: 21
P
=======================================================================
However, mod_proxy_http only sends one response:
=======================================================================
HTTP/1.1 200 OK
Date: Wed, 01 Jun 2011 15:43:50 GMT
Content-Length: 17
Connection: close
Content-Type: text/plain; charset=UTF-8
z
=======================================================================
So 2 possible problems:
- The second HTTP POST is ignored by mod_proxy_http because the previous is still pending (?)
- mod_proxy_http added "Connection: close" and bogus Content-Type header
Now what?
-
7. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
clebert.suconic Jun 1, 2011 1:11 PM (in response to greyfairer2)This will be more a netty issue. I would need Trustin to take a look on that.
At this point, as I said before, we need to make tests with mod-proxy.
Something not-tested is definitely not supported.
-
8. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
greyfairer2 Jun 1, 2011 1:11 PM (in response to greyfairer2)OK, so I added "DefaultType None" and "KeepAlive On" in my apache config, that solved the ignored POST and the additional headers.
In wireshark I now see some more requests up and down, until it breaks down again after sending something with jms-session in it:
=====================================================================
POST http://local.hornetq.qaweb.medical.barco.com:8081/hornetq1/HornetQServlet HTTP/1.1
Host: local.hornetq.qaweb.medical.barco.com:8081
Content-Length: 34
hjms-session
=====================================================================
HTTP/1.1 200 OK
Date: Wed, 01 Jun 2011 16:34:09 GMT
Content-Length: 0
=====================================================================
The result is the same: HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 104]
Can you try to set this up?
-
9. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
clebert.suconic Jun 1, 2011 1:40 PM (in response to greyfairer2)Can you create a JIRA?
I can't do it right now. But we will take a look.
-
10. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
greyfairer2 Jun 2, 2011 11:37 AM (in response to clebert.suconic)Issue created for Netty: https://issues.jboss.org/browse/NETTY-409
-
11. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
greyfairer2 Jun 22, 2011 12:29 PM (in response to greyfairer2)Any news on this? Please don't postpone this for Netty 4.0.
In general, if you are behind a Reverse mod_proxy, you cannot use the http-enabled connectionfactory at all.
You can use the servlet connectionfactory, but you can only use non-blocking calls, so only non-blocking Acknowledgement, non-blocking sends, non-transacted sessions etc.
Greets, Geert.
-
12. Re: HornetQ Servlet or HTTP via mod_proxy results in intermediate connection losses.
jvassev2 Jul 22, 2013 9:06 AM (in response to greyfairer2)Hi Geert,
I managed to put the HTTP transport behind a proxy (though not Apache mod_proxy, but a custom built one). The problem I hit and fixed had to do with the fact that
org.hornetq.core.remoting.impl.netty.NettyConnector.HttpHandler class produces requests like this:
POST http://server:ip/path/to/servlet
when in fact the request must look like:
POST /path/to/servlet
The netty transport can handle this but any intermediary would be confused and my proxy could not route such requests to the workers. The fix I applied was to replace org.hornetq.core.remoting.impl.netty.NettyConnector.HttpHandler.HttpHandler(NettyConnector) constructor with just:
url = servletPath;
It seems that HornetQ_2_3_2_FINAL still needs this fix.
I hope this helps.
Cheers, Julian