GitHub Action docker alter building context

I am running a custom action in one of my workflows with - uses: ./.github/actions/myaction/

The action.yml is defined as follows:

name: 'myaction'
description: 'myaction'
   some_input:
      description: 'some_input'
      required: true
runs:
   using: 'docker'
   image: '../../../Dockerfile'
   args:
      - ${{ inputs.some_input}}

In my Dockerfile, I COPY some files, which I have checked out in a previous workflow step. This only works if my Dockerfile is located in the root directory of my project. But I would like to move all files related to the action into the folder ./.github/actions/myaction/. If I do so, the build context is set to the action folder instead of root. Normally, one would simply run something like docker build -f ./.github/actions/myaction/ . from the root directory to build the container with correct context. Is there a way to use the action.yml file as I have it and somehow altering the build context instead? I guess GitHub can then cache the built container or something like that. Rebuilding the container in each workflow run by seems pretty bad to me.