2 Replies Latest reply on Jan 7, 2011 6:28 PM by anitaalbert

    jboss 4.0.5 - OutOfMemory in 2 hours

    anitaalbert
    anitaalbert Jan 7, 2011 11:44 AM

      Hi,

        I am a newbie to performance tuning. So sorry if these questions/descriptions sound dumb. We have an application running on jboss 4.0.5. Our application seems to crash with an Out of memory error within 2 hours. We recently did some major code revamping and a simultaneous upgrade from jdk1.5.0_11 to jdk1.6.0_23 and now we're not sure why it's crashing These are the jvm parameters -

       

      -Dsun.lang.ClassLoader.allowArraySyntax=true -Doracle.jdbc.V8Compatible=true -Xrs -Dprogram.name=run.bat -Xms1280m -Xmx1280m -Xrs -XX:MaxPermSize=192m -XX:+DisableExplicitGC -verbose:gc -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationConcurrentTime -Xloggc:D:\jboss\bin\\..\server\default\log\gc.log -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dcom.sun.management.jmxremote -Djava.endorsed.dirs=D:\jboss\bin\\..\lib\endorsed . Here's the extract of the log when it crashed -

      #
      # A fatal error has been detected by the Java Runtime Environment:
      #
      # java.lang.OutOfMemoryError: requested 894664 bytes for Chunk::new. Out of swap space?
      #
      #  Internal Error (allocation.cpp:272), pid=2000, tid=7980
      #  Error: Chunk::new
      #
      # JRE version: 6.0_23-b05
      # Java VM: Java HotSpot(TM) Server VM (19.0-b09 mixed mode windows-x86 )
      # If you would like to submit a bug report, please visit:
      #   http://java.sun.com/webapps/bugreport/crash.jsp
      #

      ---------------  T H R E A D  ---------------

      Current thread (0x5fd17000):  JavaThread "CompilerThread0" daemon [_thread_in_native, id=7980, stack(0x5ff30000,0x5ff80000)]

      Stack: [0x5ff30000,0x5ff80000],  sp=0x5ff7e5e4,  free space=313k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V  [jvm.dll+0x33e8f7]
      V  [jvm.dll+0x1065fc]
      V  [jvm.dll+0x3e0fc]
      V  [jvm.dll+0x3e393]
      V  [jvm.dll+0x2c9293]
      V  [jvm.dll+0x13cfe8]
      V  [jvm.dll+0x245f55]
      V  [jvm.dll+0xe20ab]
      V  [jvm.dll+0xe4628]
      V  [jvm.dll+0xe4f12]
      V  [jvm.dll+0xa6c7a]
      V  [jvm.dll+0xe7009]
      V  [jvm.dll+0xe7bbf]
      V  [jvm.dll+0x31e924]
      V  [jvm.dll+0x2924ac]
      C  [msvcr71.dll+0x9565]
      C  [kernel32.dll+0x2482f]


      Current CompileTask:
      C2:4077  !   org.jboss.mq.il.uil2.SocketManager$ReadTask.run()V (547 bytes)

       

      I'm attaching the entire log and also the garbage collection log. I am not able to interpret what is going wrong by looking at these logs. Could anyone please guide me as to how to interpret these logs or point me to what is going wrong? I would really appreciate it. Please let me know if any more description is needed. We are planning to deploy it for user testing in two days and a crash every two hours would be unacceptable.

        • 1. jboss 4.0.5 - OutOfMemory in 2 hours
          peterj
          peterj Jan 7, 2011 2:19 PM (in response to anitaalbert)

          This is not an "out of heap space" error. The JVM is asking the OS for more memory and the OS is saying "no." One possible cause for this is that a new thread was being created and there was insufficient memory on the system to allocate the memeory for the thread stack. Another posibility is that another file was being opened and there was no space for the file handle. Here is a post I did a while back on all of the memeory used by the JVM:

          http://www.coderanch.com/t/111262/Linux-UNIX/read-memory-usage-process-running#560894

           

          Looks like you are running a 32-bit JVM. So you could still have plenty of memory (RAM + pagefile), but run out of room in the 4GB process space.

           

          Looking at the dump stack it appears that a compiler thread was running at the time. Most likely the JIT compiler needed to allocate more memory to hold the compiled code. I don't think the compiled code is kept in the permgen (but I am not sure and I haven't found a definitive source that says one way or the other). But if it is, setting -XX:PermSize=192m should solve the problem (that will force the JVM to allocate the 192MB for the permgen at startup). If that doesn't work, you will have to reduce the heap size, or reduce the stack size for the threads. Or switch to a 64-bit JVM.

          1 of 1 people found this helpful
            • Actions
          • 2. jboss 4.0.5 - OutOfMemory in 2 hours
            anitaalbert
            anitaalbert Jan 7, 2011 6:28 PM (in response to peterj)

            Thank you so much. We added -Xss 128k and our system has been up for the last six hours without going down. We'll need to watch it over the weekend to see how it does. The server where we are going to deploy it for user testing would be using a 64 bit JVM. So I am hoping that would help as well.  Again, thanks so much for your input.

              • Actions