what I observe is that the jvm is running full once in a while. It seems there is a relation between user sessions that lost the viewstate and the memory filling. In my application there is a one-minute-interval push to update some content on the screen and that keeps on running even if the client viewstate is lost. But when the viewstate is lost I obviously see lots of errors stating the situation. Last night I had multiple users that lost the state and went to bed. The application kept on generating errors the whole night and in the moring I observe that the application is using a lot of memory.
I made a heapdump and in that I see a lot of org.richfaces.application.push.MessageData (7.456.154 instances, 298MB) and java.util.concurrent.ConcurrentLinkedQueue$Node (7.470.837 instance, 239MB).
What I assume is that the pushing keeps going but the messages are not consumed anymore and are kept in memory for ever. When I take a MessageData object and try to locate the 'nearest GC root' it traverses through and endless list of ConcurrentLinkedQueue$Node objects.
Any idea what I can do to solve this? And do you agree that it looks like a problem with push?
I've another JVM running with exactly the same application and there the MessageData is not even seen in the memory usage.
Versions: myfaces-2.1.7, tomcat-7.0.26, Richfaces-4.2.1-Final
Hello Lukáš Fryč,
I'm sorry that I don't seem to be clear about it. I'll try to explain better.
Normally everything is fine. It runs for days and even when the user is not doing anything it keeps functioning without any problem. But sometimes the memory usage starts rising enormously (in my case happens once every couple of weeks). From 150MB to even 900MB in 'CMS Old Gen' heap space. I can't be sure that there is any relation between lost viewstates and the memory hunger. It just makes sense because that is what I saw the last time. It happened both at the same time.
Normally the client reacts in some way to the push but it is normal that nobody has a view active that listens to the push topic being sent.
To replicate I guess you could somehow force a lost view state and then keep pushing events.
I was pushing an event every 20 seconds but somehow the heap dump contained a lot more. I can see where they came from based on the topicKey. 3 a minute x 60 x 10 hours = 1800 pushes max. But I see millions. Where did they come from?
Is it an idea to add an age to the push events and drop them if they are too old? Or just a max queue size.
in org.richfaces.application.push.impl.SessionImpl I see a messagesQueue that seems to be cleared based on sequenceNumber. Could it be that the sequenceNumber is not incremented somehow?
Milo van der Zee
both are pretty valid ideas.
Let's create issue which will address the problem with setting up proper tests:
If you could think about any other test cases which are worth to test, let's brainstorm there.