Codesign in Github Action prompts for keychian password

I want to use the Github Action to archive my swift project.

Here is the Github Action yaml:

name: macos_deploy

on:
  push:
    # tags: [ release* ]


jobs:
  macos_deploy:
    runs-on: self-hosted

    steps:
      - name: "Checkout Project"
        uses: actions/checkout@v2

      - name: "Import Certificate: Development"
        uses: devbotsxyz/import-signing-certificate@v1
        with:
          certificate-data: ${{ secrets.SIGNING_CERTIFICATE_P12_DATA }}
          certificate-passphrase: ${{ secrets.SIGNING_CERTIFICATE_PASSWORD }}
          keychain-password: ${{ secrets.KEYCHAIN_PASSWORD }}

      - name: "Build & Archive"
        uses: devbotsxyz/xcode-archive@v1

When I run this on the Github-hosted runner, it will hang at here:

    /usr/bin/codesign --force --sign XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -o runtime --preserve-metadata\=identifier,entitlements,flags --generate-entitlement-der /Users/runner/Library/Developer/Xcode/DerivedData/XXXXX-gfxcztatpmvmaqcfrbtnfpexcjrn/Build/Intermediates.noindex/ArchiveIntermediates/XXXXX/InstallationBuildProductsLocation/Applications/XXXXX.app/Contents/Frameworks/Sparkle.framework/Versions/B
/Users/runner/Library/Developer/Xcode/DerivedData/XXXXX-gfxcztatpmvmaqcfrbtnfpexcjrn/Build/Intermediates.noindex/ArchiveIntermediates/XXXXX/InstallationBuildProductsLocation/Applications/XXXXX.app/Contents/Frameworks/Sparkle.framework/Versions/B: replacing existing signature

So I tried to setup a self-hosted runner to debug.

Now I found that it requires me to provide the password of the keychain.

How to avoid this dialog? I think I've already provided the password at the Import Certificate: Development step.

Thanks!