Search in call stack

Question

I'm analysing memory and crash dumps of multi-threaded C++ applications.

As explained in this VisualCommunity question, I'm interested in all threads, who are not waiting and not sleeping.
Currently I get that answer using Visual Studio (Threads window) by:
1. copying all threads to an Excel sheet (column1),
2. copy the "WaitFor" filtered threads to the Excel sheet (column2),
3. copy the "Sleep" filtered threads also to the Excel sheet, (at the bottom of column2), and
4. use the ±MATCH(Column1_1;Column2_$1:Column2_$143;0) Excel worksheet function in order to get my results (filter those results on #N/A).

I'd like to know if I could do something similar (but easier) in Windbg.

Currently I can ask for a complete list of call stacks (using the Windbg command ~* kb).

Can I do a Search call stack in Windbg (most probably adding something to ~* kb command)? Are there any extended possibilities (like regular expressions) possible?

user_3622813 · Answer

the commands must be in single line remove line break at Where() if you copy paste

0:057> dx -r1 @$curprocess.Threads.Select(p=>p.Stack).Select(p=>p.Frames).Select(t=>t[1]).Where
( ( p=>p.ToDisplayString().Contains("Wait") == true ))

result of command

@$curprocess.Threads.Select(p=>p.Stack).Select(p=>p.Frames).Select(t=>t[1]).  
Where( ( p=>p.ToDisplayString().Contains("Wait") == true ))                
    [0x9dc]          : ntdll!NtWaitForMultipleObjects + 0xc [Switch To]
    [0x480]          : ntdll!NtWaitForMultipleObjects + 0xc [Switch To]
    [0xc4]           : ntdll!NtWaitForMultipleObjects + 0xc [Switch To]
    [0xae8]          : ntdll!NtWaitForSingleObject + 0xc [Switch To]
    [0xeac]          : ntdll!NtWaitForKeyedEvent + 0xc [Switch To]
    [0xf08]          : ntdll!NtWaitForMultipleObjects + 0xc [Switch To]
    [0xdd4]          : ntdll!NtWaitForSingleObject + 0xc [Switch To]
    [0xc64]          : ntdll!NtWaitForSingleObject + 0xc [Switch To]
    [0x89c]          : ntdll!NtWaitForKeyedEvent + 0xc [Switch To]
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    [0x162c]         : ntdll!NtWaitForKeyedEvent + 0xc [Switch To]

command for false condition again must be single line

0:057> dx -r1 @$curprocess.Threads.Select(p=>p.Stack).Select(p=>p.Frames).Select(t=>t[1]).Where  
( ( p=>p.ToDisplayString().Contains("Wait") == false ))

result

@$curprocess.Threads.Select(p=>p.Stack).Select(p=>p.Frames).Select(t=>t[1]). 
Where( ( p=>p.ToDisplayString().Contains("Wait") == false ))                
    [0x208]          : ntdll!NtRemoveIoCompletion + 0xc [Switch To]
    [0x3ec]          : ntdll!NtRemoveIoCompletion + 0xc [Switch To]
    [0xadc]          : user32!NtUserGetMessage + 0xc [Switch To]
    [0x1794]         : ntdll!NtDelayExecution + 0xc [Switch To]
    [0xe78]          : ntdll!NtRemoveIoCompletion + 0xc [Switch To]
    [0x1164]         : ntdll!DbgUiRemoteBreakin + 0x3c [Switch To]

a comparison of output between !busy from pde as suggested by Lieven versus inbuilt command slightly modified to exclude "RemoveIo" and "Wait" stacks

0:037> dx @$curprocess.Threads.Select(p=>p.Stack).Select(p=>p.Frames).Select(t=>t[1]).Where(    
(p=>p.ToDisplayString().Contains("Wait") != true)).Where(p=>p.ToDisplayString().   
Contains("Remove") != true)

results

@$curprocess.Threads.Select(p=>p.Stack).Select(p=>p.Frames).Select(t=>t[1]).  
Where( (p=>p.ToDisplayString().Contains("Wait") !=   
true)).Where(p=>p.ToDisplayString().Contains("Remove") != true)                
    [0xd78]          : user32!NtUserGetMessage + 0xc [Switch To]
    [0xe44]          : user32!NtUserGetMessage + 0xc [Switch To]
    [0x514]          : ntdll!DbgUiRemoteBreakin + 0x3c [Switch To]
0:037> !busy

#  37 TID:0d78 kb kbn kbnL kn knL kpn kPn
 # ChildEBP RetAddr  
00 1737fdd8 770ccde0 ntdll!KiFastSystemCallRet
01 1737fddc 770cce13 user32!NtUserGetMessage+0xc
xxxx
0b 1737ff24 00000000 ntdll!_RtlUserThreadStart+0x1b

   50 TID:0e44 kb kbn kbnL kn knL kpn kPn
 # ChildEBP RetAddr  
00 1fb8fa18 770ccde0 ntdll!KiFastSystemCallRet
01 1fb8fa1c 770c18d9 user32!NtUserGetMessage+0xc
xxxxxx
07 1fb8fb20 00000000 ntdll!_RtlUserThreadStart+0x1b

   53 TID:0514 kb kbn kbnL kn knL kpn kPn
 # ChildEBP RetAddr  
00 144cf778 7780f20f ntdll!DbgBreakPoint
01 144cf7a8 7748ed6c ntdll!DbgUiRemoteBreakin+0x3c
xxxxxxx
05 144cf848 00000000 ntdll!_RtlUserThreadStart+0x1b

Threads: 3 of 54
Frames : 1
Command: knL
Mode   : Basic

user_52598 · Answer

The PDE extension from Andrew Richards adds a !busy command

  !busy          - Equivalent of ~*knL but only displays stacks that are at least
                   'depth' frames deep (default depth is 1) and are not waiting for:-
                    ~ ntdll!NtWaitFor*
                    ~ ntdll!ZwWaitFor*
                    ~ ntdll!NtRemoveIoCompletion
                    ~ ntdll!ZwRemoveIoCompletion
                    ~ ntdll!NtReplyWaitReceivePort
                    ~ ntdll!ZwReplyWaitReceivePortEx

Problematic to find a download for it but you can ping Andrew through the Channel 9 link. He usually responds pretty quickly.

user_480982 · Answer

There's also the !findstack command. Unfortunately it needs a symbol as parameter, so wildcards or parts of names will not work.

Example:

0:001> !findstack wow64cpu!WaitForMultipleObjects32
Thread 001, 1 frame(s) match
        * 01 0000000006c0eb10 0000000073e0d286 wow64cpu!WaitForMultipleObjects32+0x3b

Thread 003, 1 frame(s) match
        * 01 000000000bb3ebf0 0000000073e0d286 wow64cpu!WaitForMultipleObjects32+0x3b

Search in call stack

3 Answers