Adding a short delay inside a loop prevents it from looping indefinitely. Why?

While using the .NET async/await API I ran into a curiosity: a loop was ignoring a delay used as a timeout, until I added a short delay inside the loop. How does this work? Not the most intuitive behavior!

Full program:

using System;
using System.Threading.Tasks;

public class Program
{
    public static void Main(String[] args)
    {
        Task.Run(async () =>
        {
            await Task.WhenAny(Loop(), Task.Delay(TimeSpan.FromSeconds(1)));
            Console.WriteLine("Timed out!");
        })
        .Wait();
    }

    public static async Task Loop()
    {
        while(true)
        {
            // Commenting this out makes the code loop indefinitely!
            await Task.Delay(TimeSpan.FromMilliseconds(1));

            // This doesn't matter.
            await DoWork();
        }
    }

    public static async Task DoWork()
    {
        await Task.CompletedTask;
    }
}

Background

The actual program has while(!done) but due to a bug done is never set to true. The loop makes many await calls. The Task.WhenAny call is in a unit test to prevent Loop() from hanging. If I introduce a bug on purpose most of the time the test indeed times out, but sometimes it still just hangs.

Suggested workaround that doesn't require Task.Delay in Loop()

bool completedOnTime = Task.Run(() => Loop()).Wait(TimeSpan.FromSeconds(1));

This will start a new thread executing the Loop() method.

Related questions

When would I use Task.Yield()?