Resource not found when navigating to path with Laravel/Inertia

I have deployed a web app with Laravel 9, Inertia, Vue3 and Apache2 on a self-hosted Ubuntu server. By calling mydomain.de I can visit the start page easily. But when I navigate to any path of the domain, for example mydomain.de/settings, I get the following error:

Not Found - The requested URL was not found on this server
Apache/2.4.41 (Ubuntu) Server at mydomain.de Port 443

I found out that the paths work when I call mydomain.de/index.php/settings for example, but then the URL reads like this

mydomain.de/index.php/index.php/settings

Locally the routing works perfectly.

AllowOverride All is set in my Apache config.

rewrite.load mod is enabled.

Apache service has been restarted multiple times.

I have also tried different .htaccess files in root folder and public folder.

Here are my files:

/etc/apache2/sites-available/mydomain.de.conf

Listen 8081
<VirtualHost *:80, *8081>

    ServerAdmin name@mydomain.de
    ServerName mydomain.de
    ServerAlias www.mydomain.de
    DocumentRoot /var/www/mydomain/public

    <Directory /var/www/mydomain/public>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride All
        Require all granted
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    Redirect "/" "https://mydomain.de"
    RewriteEngine on
    RewriteCond %{SERVER_NAME} =www.mydomain.de [OR]
    RewriteCond %{SERVER_NAME} =mydomain.de
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R]
</VirtualHost>

/var/www/mydomain/public/.htaccess

<IfModule mod_rewrite.c>
<IfModule mod_negotiation.c>
    Options -MultiViews -Indexes
</IfModule>

Options +FollowSymLinks

RewriteEngine On
RewriteBase /mydomain/public/

# Handle Authorization Header
RewriteCond %{HTTP:Authorization} .
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

# Redirect Trailing Slashes If Not A Folder...
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} (.+)/$
RewriteRule ^ %1 [L,R=301]

# Send Requests To Front Controller...
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ index.php [L]
</IfModule>

routes/web.php

<?php

  use Illuminate\Support\Facades\Route;
  use Inertia\Inertia;

  Route::get('/', function () {
    return Inertia::render('Start', [
      'title' => 'Start'
    ]);
  });

  Route::get('/settings', function () {
    return Inertia::render('Settings', [
      'title' => 'Einstellungen'
    ]);
  });

What am I doing wrong? If there is any other file I can add to make my setup clearer, do not hesitate to tell me.

Any help is appreciated.