Waiting on a Task with a OnlyOnFaulted Continuation causes an AggregateException

I have some simple code as a repro:

var taskTest = Task.Factory.StartNew(() =>
{
    System.Threading.Thread.Sleep(5000);

}).ContinueWith((Task t) =>
{
    Console.WriteLine("ERR");
}, TaskContinuationOptions.OnlyOnFaulted);

try
{
    Task.WaitAll(taskTest);
}
catch (AggregateException ex)
{
    foreach (var e in ex.InnerExceptions)
        Console.WriteLine(e.Message + Environment.NewLine + e.StackTrace);
}

However, I'm getting an unexpected TaskCanceledException being thrown in the try catch block (it's in the AggregateException InnerExceptions object). "A task was canceled".

Why am I getting this exception? The Continuation for the task never fires, there was no exception generated by it, yet I still get the aggregate exception when waiting....

I'm hoping someone can explain how this makes sense to me :)