We have the "Too many open files" error 3 times per week in JBoss 4.2.3 GA.

I think that it relates to JBoss WS because everytime logs start with records which you can see below.

Could you help us?

Ubuntu 8.04.3 LTS
ulimit -n 65536

Apache/2.2.8 (Ubuntu)
mod_proxy_ajp

 

java version "1.6.0_16"
Java(TM) SE Runtime Environment (build 1.6.0_16-b01)
Java HotSpot(TM) 64-Bit Server VM (build 14.2-b01, mixed mode)

 

JBoss 4.2.3 GA
JBoss WS 3.0.5 GA (WS-Security)

2009-12-28

2009-12-28 02:33:37,073 ERROR [org.jboss.wsf.stack.jbws.RequestHandlerImpl] Error processing web service request
org.jboss.ws.WSException: java.io.FileNotFoundException: /home/jboss/jboss-4.2.3.GA/server/default/data/wsdl/AccountManagementEJBModule.jar/AccountManagement
ServiceBeanService4651591057440438600.wsdl (Too many open files)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleWSDLRequest(RequestHandlerImpl.java:606)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.doGet(RequestHandlerImpl.java:153)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleHttpRequest(RequestHandlerImpl.java:135)
        at org.jboss.wsf.common.servlet.AbstractEndpointServlet.service(AbstractEndpointServlet.java:85)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:437)
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:366)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.FileNotFoundException: /home/jboss/jboss-4.2.3.GA/server/default/data/wsdl/AccountManagementEJBModule.jar/AccountManagementServiceBeanService4651591057440438600.wsdl (Too many open files)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:106)
        at org.jboss.net.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:105)
        at java.net.URL.openStream(URL.java:1010)
        at org.jboss.ws.core.server.WSDLRequestHandler.getDocumentForPath(WSDLRequestHandler.java:81)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleWSDLRequestFromServletContext(RequestHandlerImpl.java:633)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleWSDLRequest(RequestHandlerImpl.java:588)
        ... 22 more
2009-12-28 02:33:37,078 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/AccountWebService].[AccountManagementServiceBean]] Servlet.se
rvice() for servlet AccountManagementServiceBean threw exception
java.io.FileNotFoundException: /home/jboss/jboss-4.2.3.GA/server/default/data/wsdl/AccountManagementEJBModule.jar/AccountManagementServiceBeanService46515910
57440438600.wsdl (Too many open files)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:106)
        at org.jboss.net.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:105)
        at java.net.URL.openStream(URL.java:1010)
        at org.jboss.ws.core.server.WSDLRequestHandler.getDocumentForPath(WSDLRequestHandler.java:81)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleWSDLRequestFromServletContext(RequestHandlerImpl.java:633)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleWSDLRequest(RequestHandlerImpl.java:588)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.doGet(RequestHandlerImpl.java:153)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleHttpRequest(RequestHandlerImpl.java:135)
        at org.jboss.wsf.common.servlet.AbstractEndpointServlet.service(AbstractEndpointServlet.java:85)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:437)
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:366)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
        at java.lang.Thread.run(Thread.java:619)
2009-12-28 02:33:37,166 ERROR [org.jboss.ws.core.jaxws.SOAPFaultHelperJAXWS] SOAP request exception
org.jboss.ws.WSException: Cannot open: file:/home/jboss/jboss-4.2.3.GA/server/default/data/wsdl/AccountManagementEJBModule.jar/AccountManagementServiceBeanSe
rvice4651591057440438600.wsdl
        at org.jboss.ws.core.jaxws.handler.MessageContextJAXWS.setOperationMetaData(MessageContextJAXWS.java:148)
        at org.jboss.ws.core.server.ServiceEndpointInvoker.invoke(ServiceEndpointInvoker.java:176)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.processRequest(RequestHandlerImpl.java:474)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleRequest(RequestHandlerImpl.java:295)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.doPost(RequestHandlerImpl.java:205)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleHttpRequest(RequestHandlerImpl.java:131)
        at org.jboss.wsf.common.servlet.AbstractEndpointServlet.service(AbstractEndpointServlet.java:85)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:437)
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:366)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
        at java.lang.Thread.run(Thread.java:619)
2009-12-28 02:33:38,554 WARN  [org.jboss.deployment.scanner.URLDeploymentScanner] Scan URL, caught java.io.IOException: Could not list directory '/home/jboss/jboss-4.2.3.GA/server/default/deploy', reason unknown

 

2009-12-31

2009-12-31 02:31:33,053 ERROR [org.jboss.wsf.stack.jbws.RequestHandlerImpl] Error processing web service request
org.jboss.ws.WSException: java.io.FileNotFoundException: /home/jboss/jboss-4.2.3.GA/server/default/data/wsdl/AccountManagementEJBModule.jar/AccountManagement
ServiceBeanService6919894052436294719.wsdl (Too many open files)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleWSDLRequest(RequestHandlerImpl.java:606)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.doGet(RequestHandlerImpl.java:153)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleHttpRequest(RequestHandlerImpl.java:135)
        at org.jboss.wsf.common.servlet.AbstractEndpointServlet.service(AbstractEndpointServlet.java:85)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:437)
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:366)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.FileNotFoundException: /home/jboss/jboss-4.2.3.GA/server/default/data/wsdl/AccountManagementEJBModule.jar/AccountManagementServiceBeanService6919894052436294719.wsdl (Too many open files)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:106)
        at org.jboss.net.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:105)
        at java.net.URL.openStream(URL.java:1010)
        at org.jboss.ws.core.server.WSDLRequestHandler.getDocumentForPath(WSDLRequestHandler.java:81)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleWSDLRequestFromServletContext(RequestHandlerImpl.java:633)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleWSDLRequest(RequestHandlerImpl.java:588)
        ... 22 more
2009-12-31 02:31:33,054 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/AccountWebService].[AccountManagementServiceBean]] Servlet.service() for servlet AccountManagementServiceBean threw exception
java.io.FileNotFoundException: /home/jboss/jboss-4.2.3.GA/server/default/data/wsdl/AccountManagementEJBModule.jar/AccountManagementServiceBeanService6919894052436294719.wsdl (Too many open files)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:106)
        at org.jboss.net.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:105)
        at java.net.URL.openStream(URL.java:1010)
        at org.jboss.ws.core.server.WSDLRequestHandler.getDocumentForPath(WSDLRequestHandler.java:81)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleWSDLRequestFromServletContext(RequestHandlerImpl.java:633)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleWSDLRequest(RequestHandlerImpl.java:588)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.doGet(RequestHandlerImpl.java:153)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleHttpRequest(RequestHandlerImpl.java:135)
        at org.jboss.wsf.common.servlet.AbstractEndpointServlet.service(AbstractEndpointServlet.java:85)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:437)
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:366)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
        at java.lang.Thread.run(Thread.java:619)
2009-12-31 02:31:33,126 ERROR [org.jboss.ws.core.jaxws.SOAPFaultHelperJAXWS] SOAP request exception
org.jboss.ws.WSException: Cannot open: file:/home/jboss/jboss-4.2.3.GA/server/default/data/wsdl/AccountManagementEJBModule.jar/AccountManagementServiceBeanSe
rvice6919894052436294719.wsdl
        at org.jboss.ws.core.jaxws.handler.MessageContextJAXWS.setOperationMetaData(MessageContextJAXWS.java:148)
        at org.jboss.ws.core.server.ServiceEndpointInvoker.invoke(ServiceEndpointInvoker.java:176)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.processRequest(RequestHandlerImpl.java:474)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleRequest(RequestHandlerImpl.java:295)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.doPost(RequestHandlerImpl.java:205)
        at org.jboss.wsf.stack.jbws.RequestHandlerImpl.handleHttpRequest(RequestHandlerImpl.java:131)
        at org.jboss.wsf.common.servlet.AbstractEndpointServlet.service(AbstractEndpointServlet.java:85)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:182)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:437)
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:366)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
        at java.lang.Thread.run(Thread.java:619)