Home.... | log in.... | Pub my question! | Sign-up....

Java-index >> Core >> Monitoring & Management

jstack - error occurred during stack walking

Trying to use jstack to debug a deadlock on Solaris 8, using jdk/jre 1.5.0-08, I get the following error on stderr:

$ sudo -u apache jstack 17453 > stack

Attaching to process ID 17453, please wait...

Debugger attached successfully.

Client compiler detected.

JVM version is 1.5.0_08-b03

sun.jvm.hotspot.utilities.AssertionFailure: Expected raw sp likely got real sp, value was 0xfffffffe

at sun.jvm.hotspot.utilities.Assert.that(Assert.java:15)

at sun.jvm.hotspot.runtime.sparc.SPARCFrame.<init>(SPARCFrame.java:214)

at sun.jvm.hotspot.runtime.sparc.SPARCFrame.senderWithPCAdjustment(SPARCFrame.java:981)

at sun.jvm.hotspot.runtime.sparc.SPARCFrame.sender(SPARCFrame.java:484)

at sun.jvm.hotspot.runtime.Frame.sender(Frame.java:184)

at sun.jvm.hotspot.runtime.Frame.realSender(Frame.java:189)

at sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:102)

at sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:134)

at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:231)

at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:50)

at sun.jvm.hotspot.tools.JStack.run(JStack.java:41)

at sun.jvm.hotspot.tools.Tool.start(Tool.java:204)

at sun.jvm.hotspot.tools.JStack.main(JStack.java:58)

Any thoughts as to what I am doing wrong are appreciated.

[1401 byte] By [jgouldinga] at [2007-10-3 5:15:32]

«« Is there a Java's equivalent of .NET's DataSet?
»» Parsing a file - each line of file has integers, flooats,doubles , strings

# 1

It is not your mistake. Some times the java vm thread stacks are not in walkable state. So jstack got a bad sp and hence it asserted.-Swamy

swamyva at 2007-7-14 23:22:07 >

top of Java-index,Core,Monitoring & Management...

# 2

> It is not your mistake. Some times the java vm thread

> stacks are not in walkable state. So jstack got a bad sp

> and hence it asserted.

Thanks for the information. However, this seems to be a fairly reproducible error. Even worse, I get a similar error when I run the same program on a AMD64 box.

I am trying to debug a potential deadlock issue, and hoping that jstack will show me why things get hung up. I get the feeling that the stack that cannot be walked has an important relation to my problem.

jgouldinga at 2007-7-14 23:22:07 >

top of Java-index,Core,Monitoring & Management...

# 3

If you are able to rerun your program on Solaris-10 (or above)

then you could use pstack(1).

pstack is useful Solaris utility, it prints all stack traces of the process.

It is also very useful in deadlocks detection.

pstack understands java frames starting from Solaris-10 and JDK 5.0 and prints C+Java mixed mode stack traces.

Please, note, that on amd64 boxes you need to run:

/usr/bin/i86/pstack - to inspect a 32-bit process

/usr/bin/pstack - to inspect a 64-bit process

Good luck!

Serguei

ss45998a at 2007-7-14 23:22:07 >

top of Java-index,Core,Monitoring & Management...

# 4

Thanks for the good information. However, our target platform is Solaris 8 (needed for some of our customers). That means we only keep Solaris 8 around, to make sure everything is compatible.

The AMD64 boxes are actually running Linux (kernel 2.6.9). I don't suppose there is a Linux equivalent of pstack?

jgouldinga at 2007-7-14 23:22:07 >

top of Java-index,Core,Monitoring & Management...

# 5

You can run pstack on Solaris-8, but it will not show java names - only native frames will have function names. It may help to understand what is going on.

There is some work to port pstack on Linux (I saw this also several years ago)..

For example:

http://www.linuxcommand.org/man_pages/pstack1.html

http://sourceforge.net/projects/lsstack/

Serguei

ss45998a at 2007-7-14 23:22:07 >

top of Java-index,Core,Monitoring & Management...

# 6

Try sending ctrl-break signal to the java to get the thread dump or

try 6.0 java and jstack. The jstack on 6.0 uses dynamic attach which is similar to sending ctrl-break signal to the process. Not able to stack

walk is jstack problem due to its limitations. It is not related to your

java problem.

swamyva at 2007-7-14 23:22:07 >

top of Java-index,Core,Monitoring & Management...

# 7

I actually started out using the SIGQUIT method, but my program has the stderr stream redirected to a log file, and the signal appears to be lost/ignored/sent somewhere else... see this thread for an example: http://forum.java.sun.com/thread.jspa?threadID=568811&tstart=255

jgouldinga at 2007-7-14 23:22:07 >

top of Java-index,Core,Monitoring & Management...

# 8

If stderr is redirected the SIGQUIT is not ignored. The output should be in the redirected log file or some place. Did you check the log file? In 6.0 jstack should work better. Try it. If you do not see any output please let us know.

swamyva at 2007-7-14 23:22:07 >

top of Java-index,Core,Monitoring & Management...

# 9

> The output should be in the redirected log file or

> some place.

I definitely checked the log file. Does the SIGQUIT output go to stderr or to stdout? Unfortunately, stdout is lost, and only stderr is logged (System.err.println does work).

I will try using one of the beta JDK/JREs if these problems continue. I just can't ship them as production, for obvious reasons. :-D

jgouldinga at 2007-7-14 23:22:07 >

top of Java-index,Core,Monitoring & Management...

# 10

VM output is going to stdout. To redirect it to stderr use vm option -XX:+DisplayVMOutputToStderr.

swamyva at 2007-7-14 23:22:07 >

top of Java-index,Core,Monitoring & Management...

# 11

Awesome! Very useful information, thank you!Is there a list of these XX options? I know that java -X will list some options, and I found this page: http://java.sun.com/docs/hotspot/VMOptions.htmlbut neither of these list the option you suggested.

jgouldinga at 2007-7-14 23:22:08 >

top of Java-index,Core,Monitoring & Management...

# 12

This is undocumented option. These options subject to changewithout notice. http://blogs.sun.com/watt/resource/jvm-options-list.htmlThis is very useful link.

swamyva at 2007-7-14 23:22:08 >

top of Java-index,Core,Monitoring & Management...

Monitoring & Management

Core New Topic

Core Hot Topic