2 Replies Latest reply on May 28, 2010 5:51 PM by henk53

Jstack fails for JBoss with NoSuchSymbolException

atijms May 28, 2010 6:17 AM

Hi,

I tried to make a stack trace from a JBoss AS 5.1 process running on 64 bits Debian Lenny and Sun JDK 6U19. I however got an exception:

jstack -F 12087
Attaching to process ID 12087, please wait...
sun.jvm.hotspot.debugger.NoSuchSymbolException: Could not find symbol "gHotSpotVMTypeEntryTypeNameOffset" in any of the known library names (libjvm.so, libjvm_g.so, gamma_g)
    at sun.jvm.hotspot.HotSpotTypeDataBase.lookupInProcess(HotSpotTypeDataBase.java:390)
    at sun.jvm.hotspot.HotSpotTypeDataBase.getLongValueFromProcess(HotSpotTypeDataBase.java:371)
    at sun.jvm.hotspot.HotSpotTypeDataBase.readVMTypes(HotSpotTypeDataBase.java:102)
    at sun.jvm.hotspot.HotSpotTypeDataBase.<init>(HotSpotTypeDataBase.java:85)
    at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:568)
    at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:494)
    at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:332)
    at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
    at sun.tools.jstack.JStack.main(JStack.java:84)
Debugger attached successfully.
jstack requires a java VM process/core!

ps shows the following:

jboss    12056  0.0  0.0  10156  1344 pts/0    S    09:33   0:00 /bin/sh /opt/jboss/bin/run.sh -b 0.0.0.0
jboss    12087 38.3 62.7 7573264 3962268 pts/0 Sl   09:33  51:22 /opt/jdk/bin/java -Dprogram.name=run.sh -server -Xms1542m -Xmx4626m -XX:MaxPermSize=1542m -XX:+UseCompressedOops -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.net.preferIPv4Stack=true -Djava.endorsed.dirs=/opt/jboss/lib/endorsed -classpath /opt/jboss/bin/run.jar:/opt/jdk/lib/tools.jar org.jboss.Main -b 0.0.0.0

At the moment JBoss AS itself is running fine. Using jstack without the -F option also doesn't work.

Anyone has any idea of what could be wrong here?

1. Re: Jstack fails for JBoss with NoSuchSymbolException

atijms May 28, 2010 8:02 AM (in response to atijms)

Interestingly, when I startup JBoss AS on my localhost via Eclipse, and run the jstack command without -F, I get the same error:

jstack 17244
17244: Unable to open socket file: target process not responding or HotSpot VM not loaded
The -F option can be used when the target process is not responding

However...

At that exact moment I see the stack appearing in my Eclipse Console view! So the stack dump actually is being done, but somehow jstack gets confused.

I btw also tried a different JDK version (JDK6U20) and tried without the compressed oops option, but this didn't help. JDK 6U20 on localhost did give different errors though, a huge list of exceptions like this:

Thread 17288: (state = BLOCKED)
Error occurred during stack walking:
sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.execute(LinuxDebuggerLocal.java:152)
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet(LinuxDebuggerLocal.java:466)
    at sun.jvm.hotspot.debugger.linux.LinuxThread.getContext(LinuxThread.java:65)
    at sun.jvm.hotspot.runtime.linux_amd64.LinuxAMD64JavaThreadPDAccess.getCurrentFrameGuess(LinuxAMD64JavaThreadPDAccess.java:92)
    at sun.jvm.hotspot.runtime.JavaThread.getCurrentFrameGuess(JavaThread.java:256)
    at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:218)
    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
    at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
    at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
    at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
    at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
    at sun.tools.jstack.JStack.main(JStack.java:84)
Caused by: sun.jvm.hotspot.debugger.DebuggerException: get_thread_regs failed for a lwp
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.getThreadIntegerRegisterSet0(Native Method)
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.access$800(LinuxDebuggerLocal.java:51)
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$1GetThreadIntegerRegisterSetTask.doit(LinuxDebuggerLocal.java:460)
    at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal$LinuxDebuggerLocalWorkerThread.run(LinuxDebuggerLocal.java:127)

2. Re: Jstack fails for JBoss with NoSuchSymbolException

henk53 May 28, 2010 5:51 PM (in response to atijms)

This almost certainly happens when you execute jstack as root. As shown in the output of ps, JBoss runs via user jboss. Simply do a "su jboss" first and then execute jstack again. The exact same holds for jmap.

I think the message given by these commands should be MUCH clearer though, but that's a JDK problem and not a JBoss problem.
Actions

Go to original post