We're seeing an issue where the performance of our application would degrade over time, to the point where its unresponsive. Based on what I see, I think it might be related to how we're configuring keep-alive, and I'm wondering where I can find more information about configuring keep-alive in JBoss.
- The application's performance would degrade over the day
- eventually we will get a message saying "all 200 threads are busy, please increase the maximum thread count"
- at this point, doing a thread dump (using "kill -3") shows many threads in the following state:
"TP-Processor200" daemon prio=1 tid=0x0000002b31e372c0 nid=0x651a runnable [0x000000005408e000..0x000000005408ec30]
at java.net.SocketInputStream.socketRead0(Native Method)
- locked <0x0000002b082abec0> (a java.io.BufferedInputStream)
- when I look at the "connection scoreboard" in the JBoss web console, I see 200 connections marked with a "K" for "keep alive", also these connections look like they've been around for a long time (around 7,000,000ms, which will be 2 hours), long after we've killed all the clientsd
Based on what I saw above and some googling, I'm thinking this might be related to how we're configuring keep alive. I haven't been able to go further than this, because I don't know much about JBoss configuration. Does anybody have any information about how keep alive is being configured in JBoss?
We are using JBoss version 4.0.5 GA
It really looks like a memory leak trouble. If you run a jboss instance without any application it won't happen. So your application is causing it. Is there any Thread or ThreadPool on it? can you describe about your architecture? To detect memory leaks you should make profile of your application. I like to use http://www.eclipse.org/mat/ , But there is a lot of others tools ( yourkit ... ). Take a dump of your heap with jmap and check your hprof file on your profile tool.