Next.js default 404 page is not shown for non-existing pages

I'm creating a next.js app.

And when I enter a non-existing URL, instead of seeing the default 404 page, I see this message:

Application error: a client-side exception has occurred (see the browser console for more information).

And when I open the console, all I see is this line repeating:

on-demand-entries-client.js?4540:45  
 GET https://localhost:3000/non-existing-url 404    
eval    @   on-demand-entries-client.js?4540:45    
eval    @   websocket.js?f03c:39     
handleMessage   @   websocket.js?f03c:38

And here are the files in my pages directory. I have no custom _error.js page or 404.js page:

I'm stuck as there is no information whatsoever to tell me what's wrong in the console, and I have not created custom error pages.

What should I do?