Process.MainWindowHandle is non-zero in .NET Framework but zero in .NET Core unless debugging

Question

I am facing an odd behaviour of the Process class when accessing MainWindowHandle in .NET Core (3.1).

Consider the following function:

bool TestProcess()
{
    var process = Process.Start("notepad");
    try
    {
        for (var attempt = 0; attempt < 20; ++attempt)
        {
            process?.Refresh();
            if (process?.MainWindowHandle != IntPtr.Zero)
            {
                return true;
            }

            Thread.Sleep(100);
        }

        return false;
    }
    finally
    {
        process?.Kill();
    }
}

If the function is run in .NET Framework, it returns true as I would expect. However, when using .NET Core (3.1 in my case), it returns false instead.

Now the part that is puzzling me even more:

If I set a breakpoint anywhere after the process is started but before the MainWindowHandle property is read (or if I simply step over the code at least once between those lines), the function returns true.
If I set a breakpoint after the MainWindowHandle property is read, the function will return false. At that point, it does not matter anymore if I step over the code, set more breakpoints, etc.; the result is always false.

What could be going on and how could I fix it?

A few more details that may or may not be relevant:

The same issue can occur with other processes as long as they have a GUI (I originally discovered it with a WPF app). Try dfrgui instead for example.
Some processes such as calc seem to spawn a separate process for the actual GUI, so the behaviour changes slightly:
- In .NET Core, the function still returns false, but the GUI remains open, and the breakpoint trick does not work anymore.
- In .NET Framework, an InvalidOperationException is thrown in the MainWindowHandle line due to the process having exited already.
I am using Visual Studio 2019 (16.4.5), ReSharper 2019.3.2, .NET Core SDK 3.1.101 and Windows 10 (Build 18363).
The breakpoint trick also works in Rider (2019.3.3). However, only if you leave enough time for the main window to appear before resuming the program execution. This might also be the case in Visual Studio, but the IDE reacts too slowly for me to test.

I am guessing that the debugger is altering the behaviour of the program somehow; perhaps accidentally when listing all the process properties, or perhaps it has to do with the threads it uses. But how exactly? Could I replicate that same behaviour?

Some things that I already tried but did not work:

Increasing the number of attempts or the sleep time between attempts
Reordering the Refresh(), Thread.Sleep() and MainWindowHandle lines
Replacing the Thread.Sleep() call with await Task.Delay() (and making the function async)
Start the process with UseShellExecute set to true / false explicitly
Using [MTAThread] or [STAThread] attributes
Printing all the process properties via reflection after it is created / refreshed
- Even if this did not work, perhaps the debugger reads those properties in a different way / order, so this might still be why debugging makes a difference.

As a bit of background, I came across this issue when I added UI tests to my WPF application using FlaUI (see the issue I created). I am quite sure now that it is not a problem of the library itself; it just happens to use Process.MainWindowHandle for some of its methods.

user_1712861 · Answer

It turns out this was caused by an issue with .NET Core itself. The MainWindowHandle property would not be re-evaluated after the first attempt, no matter if it returned IntPtr.Zero.

When setting breakpoints, the only thing I was achieving was to delay the moment in which MainWindowHandle was being read. I could have achieved the same with a longer Thread.Sleep() call before that. In fact, 100 milliseconds was actually enough for Notepad in my case, but for the original WPF app I was testing I need about 1 second. Perhaps I was being too wary about debuggers in general.

I already submitted a pull request to fix this. In the meantime, if anybody is also affected by something similar, I would recommend replacing any Refresh() call with process = Process.GetProcessById(process.Id). This will return a new Process instance pointing to the same process, and hence the MainWindowHandle property can be re-evaluated without issues.

In my original example, it would look like this (reordered a bit to avoid an initial instance creation):

bool TestProcess()
{
    var process = Process.Start("notepad");
    try
    {
        for (var attempt = 0; attempt < 20; ++attempt)
        {
            if (process?.MainWindowHandle != IntPtr.Zero)
            {
                return true;
            }

            Thread.Sleep(100);
            process = Process.GetProcessById(process.Id);
        }

        return false;
    }
    finally
    {
        process?.Kill();
    }
}

Process.MainWindowHandle is non-zero in .NET Framework but zero in .NET Core unless debugging

1 Answers