Programmatic deadlock detection in java

You can do this programmatically using the ThreadMXBean that ships with the JDK:

ThreadMXBean bean = ManagementFactory.getThreadMXBean();
long[] threadIds = bean.findDeadlockedThreads(); // Returns null if no threads are deadlocked.

if (threadIds != null) {
    ThreadInfo[] infos = bean.getThreadInfo(threadIds);

    for (ThreadInfo info : infos) {
        StackTraceElement[] stack = info.getStackTrace();
        // Log or store stack trace information.
    }
}

Obviously you should try to isolate whichever thread is performing this deadlock check - Otherwise if that thread deadlocks it won't be able to run the check!

Incidentally this is what JConsole is using under the covers.

One useful hint for investigation:

If you can catch the application red handed and suspect a deadlock has occurred, go and press "Ctrl-Break" in the java.exe console window (or "Ctrl-\" on Solaris/Linux). The jvm will dump the current status and stack trace of all threads, find out dead locks and precisely describe them.

It will look something like this:

Full thread dump Java HotSpot(TM) Client VM (1.5.0_09-b03 mixed mode):

"[Test Timer] Request Queue" prio=6 tid=0x13d708d0 nid=0x1ec in Object.
    wait() [0x1b00f000..0x1b00fb68]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Unknown Source)
    at library.util.AsyncQueue.run(AsyncQueue.java:138)
        - locked <0x02e70000> (a test.server.scheduler.SchedulerRequestQueue)

    ...

Found one Java-level deadlock:
=============================
"Corba service":
  waiting to lock monitor 0x13c06684 (object 0x04697d90, a java.lang.Object),
  which is held by "[Server Connection] Heartbeat Timer"
"[Server Connection] Heartbeat Timer":
  waiting to lock monitor 0x13c065c4 (object 0x0467e728, a test.proxy.ServerProxy), which is held by "Corba service"

Java stack information for the threads listed above:
===================================================
"Corba service":
    at test.proxy.ServerProxy.stopHBWatchDog(ServerProxy:695)
    - waiting to lock <0x04697d90> (a java.lang.Object)
    ...

You might want to consider IBM's MTRAT. Prevention is better than cure after all. The Multicore Software Development Kit also comes with a deadlock detection tool.

If you don't require programmatic detection you can do this via the JConsole; on the thread tab there is a "detect deadlock" button. In JDK6 this detect locks for both intrinsic monitors and j.u.c Locks

Run up the JConsole via the $JAVA_HOM/bin/jconsole command

tempus-fugit also implements it along with a programmatic thread dumping class. It's implemented using the mbean mechanism mentioned above and offers a drop in, out-of-the-box super duper solution.

In case you want it to be done in run-time you can use watchdog for that.