Elegant way to delete rows which are not referenced by other table

I have two tables (Tasks and Timeentries), which are connected by a foreign key (TimeEntries.TaskID references Tasks.ID)

Now I'd like to delete all rows from Tasks which are not referenced by the TimeEntries table. I thought that this should work:

DELETE FROM Tasks WHERE ID not IN (SELECT TaskID FROM TimeEntries)

But it affects 0 rows, even though there are a lot of unreferenced rows in the Tasks table.

What might be the problem here? Of course I could write an SP which iterates all rows, but it seems like this could be done in a one liner.

I guess this is one of those sleeptime underflow errors. Please help!