java.io.IOException: buffer overflow kills app
jonas.zuberbuehler Jan 23, 2013 5:40 AMHi
Once in a while (more or less once a week) our webapp won't load and the user is presented a blank screen. If we check the server log there are tons of the following exception:
22-01-2013 14:17:01,328 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: java.io.IOException: buffer overflow 22-01-2013 14:17:01,329 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.jboss.errai.bus.server.io.buffers.TransmissionBuffer.checkOverflow(TransmissionBuffer.java:654) 22-01-2013 14:17:01,330 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.jboss.errai.bus.server.io.buffers.TransmissionBuffer.readWait(TransmissionBuffer.java:480) 22-01-2013 14:17:01,331 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.jboss.errai.bus.server.MessageQueueImpl.poll(MessageQueueImpl.java:119) 22-01-2013 14:17:01,332 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.jboss.errai.bus.server.servlet.DefaultBlockingServlet.pollForMessages(DefaultBlockingServlet.java:145) 22-01-2013 14:17:01,333 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.jboss.errai.bus.server.servlet.DefaultBlockingServlet.doGet(DefaultBlockingServlet.java:97) 22-01-2013 14:17:01,334 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at javax.servlet.http.HttpServlet.service(HttpServlet.java:734) 22-01-2013 14:17:01,335 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) 22-01-2013 14:17:01,335 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) 22-01-2013 14:17:01,336 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) 22-01-2013 14:17:01,337 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.jboss.weld.servlet.ConversationPropagationFilter.doFilter(ConversationPropagationFilter.java:62) 22-01-2013 14:17:01,338 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) 22-01-2013 14:17:01,339 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) 22-01-2013 14:17:01,340 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at net.comerge.kcc.server.filter.LoggingFilter.doFilter(LoggingFilter.java:39) 22-01-2013 14:17:01,341 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) 22-01-2013 14:17:01,342 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) 22-01-2013 14:17:01,343 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) 22-01-2013 14:17:01,344 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) 22-01-2013 14:17:01,345 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:397) 22-01-2013 14:17:01,346 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) 22-01-2013 14:17:01,347 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) 22-01-2013 14:17:01,348 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) 22-01-2013 14:17:01,348 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) 22-01-2013 14:17:01,349 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) 22-01-2013 14:17:01,350 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) 22-01-2013 14:17:01,351 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505) 22-01-2013 14:17:01,352 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:445) 22-01-2013 14:17:01,353 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) 22-01-2013 14:17:01,354 [ajp--127.0.0.1-8009-19] (someuser) ERROR stderr: at java.lang.Thread.run(Thread.java:679)
The only solution after this happens is to kill JBoss-AS 7.1 (it won't shutdown normally) and start it once again. We looked through all our log files to find a usage pattern on how to reproduce the error but couldn't find any. It occured after a user was idle for two hours, it occured in the middle of a user action and so on.
Do you have idea what might cause that error and/or do you have any suggestions on how to debug to find the cause?
Any help is appreciated!
cheers
--jonas