Bad Gateway with Spring Boot application behind traefik 2

I started playing around with traefik v2 to use it as a reverse proxy for my infrastructure and applications in a single node docker swarm. It worked well with applications like gitlab and swarmpit, but then I ran into problems when I tried to put my own apps behind the proxy. Maybe you can help me with this.

For testing purposes I wrote a simple RESTful web service with spring boot, which justs retruns a String over a GET endpoint, but for some reason I can't reach it (well I can reach it, but only over HTTP and with my servers IP + the applications port, not over HTTPS and the host + path I specified in my traefik configuration). I keep getting a 502 Bad Gateway response and when I inspected the log of my spring boot app I saw the following exception beeing thrown:

java.lang.IllegalArgumentException: Invalid character found in method name [0x010x010xe60x0a0x000x080x000x000x000x010x000x000x000x000x000x000x010x040xe60x0a0x000x000x000x00...]. HTTP method names must be tokens
  at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:419) ~[tomcat-embed-core-9.0.55.jar!/:na]
  at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:269) ~[tomcat-embed-core-9.0.55.jar!/:na]
  at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) ~[tomcat-embed-core-9.0.55.jar!/:na]
  at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:895) ~[tomcat-embed-core-9.0.55.jar!/:na]
  at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1722) ~[tomcat-embed-core-9.0.55.jar!/:na]
  at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) ~[tomcat-embed-core-9.0.55.jar!/:na]
  at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) ~[tomcat-embed-core-9.0.55.jar!/:na]
  at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-embed-core-9.0.55.jar!/:na]
  at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) ~[tomcat-embed-core-9.0.55.jar!/:na]
  at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]

For me, it seems like the request is still encrypted when it reaches my application, also I expected traefik to route the HTTPS request in plain text, since well...isn't that the point of a reverse proxy? And if not, how do other applications decrypt the request while not having any access to the public key? They live behind the traefik proxy as well after all.

I must admit, that I'm not very knowledgeable about this kind of stuff, so I maybe get something fundamentally wrong here. Anyway, here are my configurations, maybe some of you are able to make out any errors (the domain names and email addresses are different in my files though):

Traefik compose-file

version: '3.7'

services:
  traefik:
    image: traefik:v2.2
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
    deploy:
      placement:
        constraints:
          - node.labels.traefik-public.traefik-public-certificates == true
      labels:
        - "traefik.enable=true"
        - "traefik.docker.network=traefik-public"
        - "traefik.constraint-label=traefik-public"
        - "traefik.http.middlewares.admin-auth.basicauth.usersfile=/run/secrets/traefik_users"
        - "traefik.http.middlewares.https-redirect.redirectscheme.scheme=https"
        - "traefik.http.middlewares.https-redirect.redirectscheme.permanent=true"
        - "traefik.http.routers.traefik-public-http.rule=Host(`traefik.example.com`)"
        - "traefik.http.routers.traefik-public-http.entrypoints=http"
        - "traefik.http.routers.traefik-public-http.middlewares=https-redirect"
        - "traefik.http.routers.traefik-public-https.rule=Host(`traefik.example.com`)"
        - "traefik.http.routers.traefik-public-https.entrypoints=https"
        - "traefik.http.routers.traefik-public-https.tls=true"
        - "traefik.http.routers.traefik-public-https.service=api@internal"
        - "traefik.http.routers.traefik-public-https.tls.certresolver=le"
        - "traefik.http.routers.traefik-public-https.middlewares=admin-auth"
        - "traefik.http.services.traefik-public.loadbalancer.server.port=8080"
    secrets:
      - traefik_users
    configs:
      - source: traefik
        target: /etc/traefik/traefik.toml
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - traefik-public-certificates:/certificates
    networks:
      - traefik-public
secrets:
  traefik_users:
    file: ./traefik_users.txt
configs:
  traefik:
    file: traefik.toml
volumes:
  traefik-public-certificates:
networks:
  traefik-public:
    external: true

Traefik configuration file

[global]
checkNewVersion = true
sendAnonymousUsage = false

[entryPoints]
  [entryPoints.http]
    address = ":80"
  [entryPoints.https]
    address = ":443"

[log]
  level = "INFO"
[accessLog]

[api]

[providers.docker]
  constraints = "Label(`traefik.constraint-label`, `traefik-public`)"
  exposedByDefault = false
  swarmmode = true
  network = "traefik-public"

[certificatesresolvers.le.acme]
  email = "test@mail.de"
  storage = "/certificates/acme.json"
  tlschallenge = true

Spring boot app compose-file:

version: '3.7'

services:
  demo:
    image: demo:latest
    ports:
      - "9999:8080"
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik-public
        - traefik.constraint-label=traefik-public

        - traefik.http.routers.demo-http.rule=Host(`apis.example.com`) && PathPrefix(`/demo`)
        - traefik.http.routers.demo-http.entrypoints=http
        - traefik.http.routers.demo-http.middlewares=https-redirect

        - traefik.http.routers.demo-https.rule=Host(`apis.example.com`) && PathPrefix(`/demo`)
        - traefik.http.routers.demo-https.entrypoints=https
        - traefik.http.routers.demo-https.tls=true
        - traefik.http.routers.demo-https.tls.certresolver=le
        
        - traefik.http.services.demo.loadbalancer.server.port=80
    networks:
      - traefik-public
networks:
  traefik-public:
    external: true

Spring boot app application.properties

server.port=8080
server.forward-headers-strategy=native

I tried to fix this by enabling X-Forward-Header handling, as you can see, but that didn't help.

Would be cool if someone can help me with this since I am out of my league here.

>>Update<<

So I finally found my problem. I misunderstood the loadbalancer.server.port property...I thought you must specify the port where traefik listens, but in fact you have to put the port here, where your application expects the traffic from the traefik proxy.