How to deploy multiple functions from a monorepo using Cloud Build but only one at a time

Viewed 2008

I am trying to set up a monorepo with multiple cloud functions written in Python. I am currently using Cloud Build and structure like this:

.
├── deployment
│   └── cloudbuild.yaml
├── main.py
└── requirements.txt

which with this Cloud Build YAML code deploys well:

steps:
  - name: 'gcr.io/cloud-builders/gcloud'
    args: [
      'functions', 'deploy', '$_FUNCTION_NAME',
      '--trigger-resource', '$_TRIGGER_RESOURCE',
      '--trigger-event', '$_TRIGGER_EVENT',
      '--runtime', 'python37',
      '--memory', '1024MB',
      '--region', 'europe-west1'
    ]

Now my intent is to move towards this structure:

.
├── first_function
│   ├── main.py
│   └── requirements.txt
├── second_function
│   ├── main.py
│   └── requirements.txt
└── cloudbuild.yaml

With triggers set up to watch changes within the respective subfolders, injecting the function name as env variable and deploying the correct function. This is the TF idea of setup:

resource "google_cloudbuild_trigger" "first_function_trigger" {
  project = google_project.my_project.name
  name = "trigger-first-function"
  description = "Trigger for deploying first function"

  trigger_template {
    repo_name = google_sourcerepo_repository.functions.name
    branch_name = "master"
    dir = "first_function/**"
  }

  substitutions = {
    _TRIGGER_RESOURCE = google_storage_bucket.my_bucket.name
    _TRIGGER_EVENT = "google.storage.object.finalize"
    _FUNCTION_NAME = "first_function"
  }

  filename = "cloudbuild.yaml"
}

However, here is the catch:

All arrangements, with specifying --source in the gcloud functions deploy command, just keep giving me errors such as:

ERROR: (gcloud.functions.deploy) argument --source: Provided directory does not exist

This error occurrs when I try these values:

1. --source=.
2. --source=./first_function
3. --source=./first_function/

Number three works locally when gcloud functions deploy is called from the root folder. I read about the approach of specifying the repository in GCP - but that's an extra data loading operation, no? The source code is already there - this is a trigger for changes in the repository.

When no --source is defined, this is the error I get:

ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: Build error details not available

I know that Cloud Build is a fairly young service and changes very rapidly, but is there a way now to arrange the folders or set up the cloud build YAML so that the functions get deployed correctly? I really don't want to create a separate repository for every single 100-line function.

1 Answers

I was unable to reproduce your issue with just Cloud Functions + Cloud Build. With the following structure:

.
├── cloudbuild.yaml
├── first_function
│   ├── main.py
│   └── requirements.txt
└── second_function
    ├── main.py
    └── requirements.txt

And the following cloudbuild.yaml:

steps:
  - name: 'gcr.io/cloud-builders/gcloud'
    args: [
      'functions', 'deploy', 'first_function',
      '--trigger-http',
      '--runtime', 'python37',
      '--region', 'us-central1',
      '--source', 'first_function'
    ]
  - name: 'gcr.io/cloud-builders/gcloud'
    args: [
      'functions', 'deploy', 'second_function',
      '--trigger-http',
      '--runtime', 'python37',
      '--region', 'us-central1',
      '--source', 'second_function'
    ]

I was able to deploy both functions.

Is it possible the source flag is not being set correctly?

Related