Why is custom 404 PHP error page only working online for URLs where non-existing directories are involved, no matter if the file exists or not?

My custom 404 error page only seems to work online for URLs that contain directories that don't exist. If I only have a non-existing file (in an existing directory tree), then I get - instead of my desired custom 404 page - a blank page with "File not found". Seems like, no matter if I have a non-existing file or an existing file in my URL, I only get my custom 404 page when there is a non-existing / misspelled directoy somewhere in the URL. Offline, on Xampp it works fine, though. Can anybody point me in the right direction as to what I might be doing wrong?

This is what my ".htaccess" looks like, which I put in my root directory along with my custom error page:

ErrorDocument 400 /error404.php
ErrorDocument 401 /error404.php
ErrorDocument 403 /error404.php
ErrorDocument 404 /error404.php
ErrorDocument 405 /error404.php
ErrorDocument 408 /error404.php
ErrorDocument 500 /error404.php
ErrorDocument 502 /error404.php
ErrorDocument 504 /error404.php

<IfModule mod_rewrite.c>
     RewriteEngine on
     RewriteCond %{HTTPS} !=on
     RewriteRule ^(.*)$ https://%{HTTP_HOST}/$1 [R,QSA]
</IfModule>

Update: It seems that the cause of my problems is the file extention ".php". If I call files that don't exist with an extention ".php" I get "File not found". If I use the extention ".html" in my query my custom 404 error page works alright.

example with html extention

example with php extention

Technial Support from my hosting provider suggests to have a look at my configuration where I include extentions. I am not sure where I should look to be honest. Maybe someone can narrow this down?

I have to further test this, but the following lines in my .htaccess file might do the trick:

RewriteEngine on
    RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
    RewriteRule ^.+\.php$ /non_existant_file