-
1. Re: Paging
timfox Jan 6, 2010 1:14 PM (in response to msimaan)You are not acknowledging your messages.
This has been discussed on several threads before.
-
2. Re: Paging
timfox Jan 6, 2010 1:16 PM (in response to timfox)If you haven't acked messages they will stil be in memory on the server. Therefore the server cannot depage any more messages from storage so you can consume them, since this will increase server memory above the max limit.
10K is a very small figure for max size. Make sure it is greater than ack batch size.
-
3. Re: Paging
clebert.suconic Jan 6, 2010 1:17 PM (in response to timfox)10 KIB is too little also. You should probably use something like 10 * 1024 * 1024 (10 MIB). -
4. Re: Paging
msimaan Jan 7, 2010 5:25 AM (in response to clebert.suconic)Thanks guys for your quick replies.
I read the following thread and jira issue: http://community.jboss.org/thread/19106?start=30&tstart=0 and https://jira.jboss.org/jira/browse/HORNETQ-182
So in the end, on my session is on AUTO_ACKNOWLEDGE. So the problem must was the maximum size. I changed it to 10 MB and was able to consume all the messages.
I still have one more question, I noticed that I couldn't see all the messages in Hermes though. Do you have an idea why?
The thread also convinced me of reverting from using Spring's JmsTemplate. I will replace it with my own implementation.
Thanks,
Marwan
-
5. Re: Paging
jmesnil Jan 7, 2010 5:27 AM (in response to msimaan)1 of 1 people found this helpfulHermes allows to see only messages from a queue which are in memory but not the "paged" messages (saved on the file system).
So it is expected to not be able to browse all messages if some are paged.
-
6. Re: Paging
msimaan Jan 7, 2010 5:32 AM (in response to jmesnil)But in some way this is misleading. For an end use, the paging mechanism should be transparent. My guess is that Hermes does a browse on the queues, and this can be made to read the messages paged.
Does the paging work with consume/browse + selectors?
-
7. Re: Paging
jmesnil Jan 7, 2010 5:45 AM (in response to msimaan)You're right: browsing and paging are conflicting with each other.
The whole idea behind paging messages is to not exhaust server memory by keeping all messages in memory, so we store them on file system once a given threshold size is reached.
Meanwhile, browsing a message means that the server will be left it in the queue and the client will not acknowledge it.
If we were depaging messages when the client browses the queue, this mean that eventually *all* messages could be depaged and
take server memory without being ever acknowledged by the client. This could result in an OutOfMemoryException on the server just
when browsing a huge queue and this is not something we want to allow.
Having said that, we have several enhancements planned to HornetQ 2.1 to give a bit more information on paged messages.
Paging works fine with consumers (and selectors) but is conflicting with browsers.
Hope this clarifies your issue,
jeff
-
8. Re: Paging
timfox Jan 7, 2010 6:28 AM (in response to jmesnil)It's not really a conflict.
When a message is paged, this occurs *before* a message is routed to any queues, therefore the message is not in any queues yet, so browsing is correct in not viewing that message - it really *is not* in the queue.
We've also discussed this before on other threads.
-
9. Re: Paging
msimaan Jan 7, 2010 8:27 AM (in response to timfox)If the messages are not really on the queue how does the consumer receive with a selector?
From a user point of view this is a conflict because you send a message to the queue and you can't see (browse) this message. This is not in the JMS specification at least.
To get this straight, is paging the recommended solution with HornetQ if you have a lot of queues with a lot of messages? I know it is not the default solution.
We used to work with Websphere MQ before and we didn't have this problem. My guess that messages are not stored in memory in MQ, but this is transparent to the user.
We are interested a lot in HornetQ as a standalone server due to portability advantages. So i am trying to understand how this works.
Thanks a lot for the help,
Marwan
-
10. Re: Paging
ataylor Jan 7, 2010 8:42 AM (in response to msimaan)Messages can not be consumed by a consumer if they are paged. It works like this, a message is routed via an address which will have 1 or more queues. At this point a check is made to see if paging should occur. If yes then the message is paged, if not they are delivered to the queues. At some point in time when n messages have been acknowledged then messages are depaged and put onto the queues. So if you are using selectors it is possible that a message won't get consumed even if it should be the next message delivered. The trick here is to tune your application so this doesn't happen, adjust the paging size and make sure you have enough memory to cope with this.1 of 1 people found this helpful -
11. Re: Paging
msimaan Jan 7, 2010 8:50 AM (in response to ataylor)Okay it is clear now.
But this doesn't follow the JMS standard, right?
-
12. Re: Paging
timfox Jan 7, 2010 8:54 AM (in response to msimaan)I don't follow.
What is it that you think doesn't follow the JMS spec?
-
13. Re: Paging
msimaan Jan 7, 2010 9:13 AM (in response to timfox)The fact that you cannot browse all the messages and the fact that you cannot consume any message using a selector.
What about the recommendation for paging? Is there another way?
-
14. Re: Paging
ataylor Jan 7, 2010 9:18 AM (in response to msimaan)A browser is just a snapshot of a queue at a particular point in time, if a message is paged then it wont be browsed. remember that producers and consumers are decoupled so there no requirement by the jms spec regarding timeliness of when a message should actually reach the queue.
Regarding another solution re paging, i'm not sure what you are trying to acheive. Its a trade of a server can only handle so many messages in memory before running out of it, so you either delay messages and page them or run out of memory. I am guessing that most messaging vendors will do something similar. Like i said before, you can tune paging so this doesnt occur.