2 Replies Latest reply on Sep 29, 2015 12:41 PM by arunav.sinha

    Jboss EAP 6.2 crashed silently

    arunav.sinha
    arunav.sinha Sep 28, 2015 12:39 PM

      We are using jboss-eap-6.2 in which multiple web applications are deployed in it. JDK used is jdk1.7.0_51 and linux version is SUSE Linux.

       

      From past few months we are facing issue with JBoss server, the JBoss process goes down silently without logging any error or exception in any server or application logs after running for some days. The behavior is not periodic as some time it goes down after 5 days after running continuously and sometime it goes down after 20 days. As suggested in many places we have set "-Xrs" at VM start up, but this also did not helped and Jboss crashed again with same behaviour. In order to further debug issues, we have enabled GC logs and VM thread logs. Exactly at time of crash I could see below logs.Its been observed at time of crash nothing much is happening on the server and mostly its ideal at that time.

       

      <dependency_failed type='abstract_with_unique_concrete_subtype' ctxk='com/sun/xml/bind/v2/runtime/Coordinator' x='com/sun/xml/bind/v2/runtime/unmarshaller/UnmarshallingContext' witness='com/sun/xml/bind/v2/runtime/XMLSerializer' stamp='92528.135'/>

      <dependency_failed type='abstract_with_unique_concrete_subtype' ctxk='com/sun/xml/bind/v2/runtime/Coordinator' x='com/sun/xml/bind/v2/runtime/unmarshaller/UnmarshallingContext' witness='com/sun/xml/bind/v2/runtime/XMLSerializer' stamp='92528.135'/>

      <dependency_failed type='abstract_with_unique_concrete_subtype' ctxk='com/sun/xml/bind/v2/runtime/Coordinator' x='com/sun/xml/bind/v2/runtime/unmarshaller/UnmarshallingContext' witness='com/sun/xml/bind/v2/runtime/XMLSerializer' stamp='92528.135'/>

      <dependency_failed type='abstract_with_unique_concrete_subtype' ctxk='com/sun/xml/bind/v2/runtime/Coordinator' x='com/sun/xml/bind/v2/runtime/unmarshaller/UnmarshallingContext' witness='com/sun/xml/bind/v2/runtime/XMLSerializer' stamp='92528.135'/>

       

      From initial analysis we are quite sure that issue is not related to out of memory. The JVM is started with -Xms2048m -Xmx8192m -XX:MaxPermSize=1024m -Xrs -verbose:gc -XX:+PrintGCDateStamps -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath= /logs/jbossOutOfMemory -Xrs -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput  -XX:LogFile=/logs/threadStackTrace.log

       

      We are not able to replicate the issue and not sure what's root cause of the Jboss crash. Any help would be very much appreciated.

       

      Thanks in Advance.

        • 1. Re: Jboss EAP 6.2 crashed silently
          wdfink
          wdfink Sep 28, 2015 3:10 PM (in response to arunav.sinha)

          Did you see any hs_err_pid.log files, normally you will find it in the directory where you start JBoss.

          This is a file written if the JVM crash. Also a GC.log file might give hints

            • Actions
          • 2. Re: Jboss EAP 6.2 crashed silently
            arunav.sinha
            arunav.sinha Sep 29, 2015 12:41 PM (in response to arunav.sinha)

            Hi Dieter

             

            We have setup VM parameter  -XX:ErrorFile=/logs/jbossErrorLog.log to generate the error log at defined server location, the error log is empty. I even check the bin folder location for hs_err_pid.log, it was not generated. I analyzed the GC logs, the last GC happened 2 minutes before the crash. There was no abnormal behaviour in GC log.

              • Actions