Debugging a hung process

user_401445 0
Viewed 3103

I'm trying to debug a hung java process. It is unresponsive to most of the JDK tools like jstats, jmap etc.

Here is the output of strace:

$ sudo strace -p <pid>
futex(0x7f14bb42a9d0, FUTEX_WAIT, 6090, NULL <unfinished ...>

Output of wchan:

$ sudo cat /proc/<pid>/wchan          
futex_wait_queue_me%  

$sudo strace -f -p <pid>
Process <pid> attached with 15 threads - interrupt to quit
[pid  6105] futex(0x7f14b40cb954, FUTEX_WAIT_PRIVATE, 1, NULL <unfinished ...>
[pid  6102] futex(0x7f14ba81e860, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
[pid  6101] futex(0x7f14b408aa54, FUTEX_WAIT_PRIVATE, 43, NULL <unfinished ...>
[pid  6100] futex(0x7f14b4085f54, FUTEX_WAIT_PRIVATE, 43, NULL <unfinished ...>
[pid  6074] futex(0x7f14bb42a9d0, FUTEX_WAIT, 6090, NULL

Other important information about the process:

  • It runs under cgexec.

Need pointers on how to find the root cause of this process hang issue.

strace debugging java
2 Answers

One way is to take thread dump and analyze it for some deadlock, look for blocked threads and identify what resources they are blocked on. If it is Oracle hotspot VM, then there are other few utilities you can run to check for deadlock detection.If the process is not responding, you can force the process to dump the stacktrace(thread dump) . Read more about Oracel JVM troubleshooting. https://docs.oracle.com/javase/7/docs/webnotes/tsg/TSG-VM/html/hangloop.html

user_6972130

I had the same problem in android, so I have used the strace logs to check where its stuck. But strace you have to use in live mode.

I mean you have to keep running your program with strace.

I was using the below command to get the live strace logs.

strace -f -p $(pidof <process_name>)
user_136985510
Related