Throwing exceptions from ContinueWith

I am trying to wrap the exceptions that can be thrown by an async task using ContinueWith(). If I just throw from the continuation action things seem to work, but my debugger claims the exception is unhandled. Am I doing something wrong or is this a Visual Studio problem? Is there a cleaner way to do this, or a way to work around my debugger stopping on what is ultimately a handled exception?

The test below passes and prints "caught wrapped exception as expected", but when I debug it the throw new CustomException line shows as "unhandled by user code".

var task = DoWorkAsync().ContinueWith(t => {
    throw new CustomException("Wrapped", t.Exception.InnerException);  // Debugger reports this unhandled
}, TaskContinuationOptions.OnlyOnFaulted | TaskContinuationOptions.ExecuteSynchronously);

try {
    task.Wait();
    Assert.Fail("Expected work to fail");
} catch (AggregateException ag) {
    if (!(ag.InnerException is CustomException))
        throw;
}
Console.WriteLine("Caught wrapped exception as expected");