MongoDB single node replicaset never finishing instanciating Error: "Cannot use non-local read concern until replica set is finished initializing"

I'm trying to setup a mongodb replicaset as a single node in a docker container because I need to use the transactions to watch over changes in collections.

The issue I'm having is that the replicaset seems to never finish to instanciate and the loog display the following line every seconds:

{"t":{"$date":"2022-05-19T21:31:57.421+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":9200}}

I've tried every tutorials I've found, every stackoverflow issue, they all say the same thing, the configuration should be fairly simple and work with what I have now.

Here's my configurations:

mongo-init.js (I removed the db and collection creation in hope to make it work)

const rsconf = {
    _id: "rs0",
    members: [
        {
            _id: 0,
            host: '127.0.0.1:27017',
        },
    ]
};

rs.initiate(rsconf);

Dockerfile (I use a custom image because I'm on docker windows and I can't set proper file permission on a file if I mount it in the container from the host. It's a POC security is not a concern anyway)

FROM mongo:latest
COPY mongo/mongo-conf/mongodb.key /mongodb.key
RUN chmod 600 /mongodb.key
RUN chown 999:999 /mongodb.key

# To solve "Unable to open() file /home/mongodb/.dbshell - source: https://github.com/docker-library/mongo/issues/323#issuecomment-494648458
RUN mkdir "/home/mongodb"
RUN touch "/home/mongodb/.dbshell"
RUN chown -R 999:999 /home/mongodb

ENTRYPOINT ["docker-entrypoint.sh"]

docker-compose.yml

version: '3.8'
services:
  # Mongodb database
  mongo:
    image: mongo-custom
    container_name: mongo
    command: --bind_ip_all --replSet rs0 --keyFile /mongodb.key
    #    restart: unless-stopped
    environment:
      MONGO_INITDB_ROOT_USERNAME: mogoadm
      MONGO_INITDB_ROOT_PASSWORD: mogopwd
    ports:
      - "27017:27017"
    volumes:
      - ./mongo/database/mongodb/db:/data/db
      - ./mongo/mongo-init:/docker-entrypoint-initdb.d:ro

I did read the official documentation and I see no functionnal difference, I'm really banging my head against a wall here trying to figure out what I did wrong?