Making authorised requests from Google Cloud Build

I'm trying to set up a deployment street in Google Cloud Build. To do this, I want to:

1. Run unit test
2. Deploy to Cloud Run without traffic
3. Run integration tests
4. Migrate traffic in Cloud Run

I've got this mostly set up, but my integration tests include a couple of calls to Cloud Run to validate that authenticated calls return 200 and unauthenticated return 401. The thing I'm having difficulties with is to make signed requests from Cloud Build. When deploying by hand and running integration tests, they work, but not from Cloud Build.

Ideally, I would like to use the Cloud Build Service Account for invoking Cloud Run like I usually do in AWS, but I can't find how to get access to that from the Cloud Runner. So instead, I retrieve a credentials file from Secret Manager. This credentials file is from a newly created Service Account with Cloud Run Invoker role:

steps:
  - name: gcr.io/cloud-builders/gcloud
    id: get-github-ssh-secret
    entrypoint: 'bash'
    args: [ '-c', 'gcloud secrets version access latest --secret=name-of-secret > /root/service-account/credentials.json' ]
    volumes:
      - name: 'service-account'
        path: /root/service-account
...
  - name: python:3.8.7
    id: integration-tests
    entrypoint: /bin/sh
    args:
      - '-c'
      - |-
        if [ $_STAGE != "prod" ]; then 
          python -m pip install -r requirements.txt
          python -m pytest test/integration --disable-warnings ; 
        fi
    volumes:
      - name: 'service-account'
        path: /root/service-account

For the integration tests, I've created a class called Authorizer and I have __get_authorized_header_for_cloud_build and __get_authorized_header_for_cloud_build2 as attempts:

import json
import time
import urllib
from typing import Optional

import google.auth
import requests
from google import auth
from google.auth.transport.requests import AuthorizedSession
from google.oauth2 import service_account
import jwt


class Authorizer(object):
    cloudbuild_credential_path = "/root/service-account/credentials.json"

    # Permissions to request for Access Token
    scopes = ["https://www.googleapis.com/auth/cloud-platform"]

    def get_authorized_header(self, receiving_service_url) -> dict:
        auth_header = self.__get_authorized_header_for_current_user() \
                      or self.__get_authorized_header_for_cloud_build(receiving_service_url)
        return auth_header

    def __get_authorized_header_for_current_user(self) -> Optional[dict]:
        credentials, _ = auth.default()
        auth_req = google.auth.transport.requests.Request()
        credentials.refresh(auth_req)
        if hasattr(credentials, "id_token"):
            authorized_header = {"Authorization": f'Bearer {credentials.id_token}'}
            auth_req.session.close()
            print("Got auth header for current user with auth.default()")
            return authorized_header

    def __get_authorized_header_for_cloud_build2(self, receiving_service_url) -> dict:
        credentials = service_account.Credentials.from_service_account_file(
            self.cloudbuild_credential_path, scopes=self.scopes)
        auth_req = google.auth.transport.requests.Request()
        credentials.refresh(auth_req)
        return {"Authorization": f'Bearer {credentials.token}'}

    def __get_authorized_header_for_cloud_build(self, receiving_service_url) -> dict:
        with open(self.cloudbuild_credential_path, 'r') as f:
            data = f.read()
        credentials_json = json.loads(data)

        signed_jwt = self.__create_signed_jwt(credentials_json, receiving_service_url)
        token = self.__exchange_jwt_for_token(signed_jwt)
        return {"Authorization": f'Bearer {token}'}

    def __create_signed_jwt(self, credentials_json, run_service_url):
        iat = time.time()
        exp = iat + 3600
        payload = {
            'iss': credentials_json['client_email'],
            'sub': credentials_json['client_email'],
            'target_audience': run_service_url,
            'aud': 'https://www.googleapis.com/oauth2/v4/token',
            'iat': iat,
            'exp': exp
        }
        additional_headers = {
            'kid': credentials_json['private_key_id']
        }
        signed_jwt = jwt.encode(
            payload,
            credentials_json['private_key'],
            headers=additional_headers,
            algorithm='RS256'
        )
        return signed_jwt

    def __exchange_jwt_for_token(self, signed_jwt):
        body = {
            'grant_type': 'urn:ietf:params:oauth:grant-type:jwt-bearer',
            'assertion': signed_jwt
        }
        token_request = requests.post(
            url='https://www.googleapis.com/oauth2/v4/token',
            headers={
                'Content-Type': 'application/x-www-form-urlencoded'
            },
            data=urllib.parse.urlencode(body)
        )
        return token_request.json()['id_token']

So when running locally, the __get_authorized_header_for_current_user is being used and works. When running in Cloud Build, __get_authorized_header_for_cloud_build is used. But even when temporarily disabling __get_authorized_header_for_current_user and let cloudbuild_credential_path reference to a json-file on my local pc, it keep getting 401s. Even when I give the service account from the credentials-file Owner rights. Another attempt is __get_authorized_header_for_cloud_build where I try to get the token more by myself instead of a package, but still 401.

For completeness, the integration test look somewhat like this:

class NameOfViewIntegrationTestCase(unittest.TestCase):
    base_url = "https://**.a.run.app"
    name_of_call_url = base_url + "/name-of-call"

    def setUp(self) -> None:
        self._authorizer = Authorizer()

    def test_name_of_call__authorized__ok_result(self) -> None:
        # Arrange
        url = self.name_of_call_url 

        # Act
        response = requests.post(url, headers=self._authorizer.get_authorized_header(url))

        # Arrange
        self.assertTrue(response.ok, msg=f'{response.status_code}: {response.text}')

Any idea what I'm doing wrong here? Let me know if you need any clarification on something. Thanks in advance!