Why is Visual Studio looping when debugging UnhandledException events

(This looks very similar to C# UnhandledException from another thread keeps looping, but am not trying to catch the exception here, just get the chance to log something)

I have some very simple C# code that sets up an UnhandledException event handler, then throws an exception:

class Program
{
    static void Main(string[] args)
    {
        AppDomain currentDomain = AppDomain.CurrentDomain;
        //currentDomain.UnhandledException += new UnhandledExceptionEventHandler(currentDomain_UnhandledException);

        currentDomain.UnhandledException += (sender, eventArgs) =>
            {
                var exception = (Exception) eventArgs.ExceptionObject;

                Console.WriteLine("Unhandled exception: " + exception.Message);
            };

        throw new AccessViolationException("Bleurgh");
    }
}

It behaves as I expect from the console:

Unhandled exception: Bleurgh
Unhandled Exception: System.AccessViolationException: Bleurgh
    at UnhandledExceptions.Program.Main(String[] args) in c:\code\sandbox\UnhandledExceptions\UnhandledExceptions\Program.cs:line 20

But when I attempt to debug it in Visual Studio it enters a loop, going into the event handler then dropping out to rethrow the Exception.

The same thing happens when I express the handler as a distinct static method.

Any ideas what's going on?

This is in Visual Studio 2010. EDIT: and .NET 4.