In GCP, through Cloudbuild how can I ensure that only those steps get triggered for which changes have happened in file

user_12355496 0
Viewed 3127

My issue is that, given the below yaml file, if I'm making changes for example in any file of "dir: process/cbd-bu-data", Cloud Build runs all the steps serially when triggered. This leads to wastage of time.

I want that only that step runs in cloudbuild for which the changes have been made in file of that directory. What should I do to achieve this?

Here's my cloudbuild.yaml file:

steps: 
  - args: 
      - beta
      - functions
      - deploy
      - "--runtime=python37"
      - "--trigger-http"
      - "--entry-point=process_cbd_group_data"
      - process_cbd_group_data
      - "--region=us-central1"
    dir: process/cbd-group-data
    name: gcr.io/cloud-builders/gcloud
  - args: 
      - beta
      - functions
      - deploy
      - "--runtime=python37"
      - "--trigger-http"
      - "--entry-point=process_cbd_bu_data"
      - process_cbd_bu_data
      - "--region=us-central1"
    dir: process/cbd-bu-data
    name: gcr.io/cloud-builders/gcloud
  - args: 
      - beta
      - functions
      - deploy
      - "--runtime=python37"
      - "--trigger-http"
      - "--entry-point=process_cbd_structure_data"
      - process_cbd_structure_data
      - "--region=us-central1"
    dir: process/cbd-structure-data
    name: gcr.io/cloud-builders/gcloud
google-cloud-build google-cloud-functions google-cloud-platform
3 Answers

You cannot do this from one cloudbuild. What you can do is create three different build triggers with the --included-files option. I think it is not convenient to accomplish the same thing with branches or tags as I read in the other answer. Read the documentation for more details.

Your git repository layout:

function_one/
   main.py
   cloudbuild.yaml

function_two/
   main.py
   cloudbuild.yaml

function_three/
   main.py
   cloudbuild.yaml

cloudbuild.yaml

Layout of the parent cloudbuild.yaml:

steps:
  - name: 'gcr.io/cloud-builders/gcloud'
    entrypoint: 'bash'
    args:
      - '-c'
      - |
        cloud beta builds triggers create github build_one --included-files "function_one/*" --repo-name=XXX --repo-owner=XXX --branch-pattern=$BRANCH_NAME
        cloud beta builds triggers create github build_two --included-files "function_two/*" --repo-name=XXX --repo-owner=XXX --branch-pattern=$BRANCH_NAME
        cloud beta builds triggers create github build_three --included-files "function_three/*" --repo-name=XXX --repo-owner=XXX --branch-pattern=$BRANCH_NAME

Layout of the sub cloudbuild.yaml:

steps: 
  - args: 
      - functions
      - deploy
      - "--runtime=python37"
      - "--trigger-http"
      - "--entry-point=process_cbd_group_data"
      - process_cbd_group_data
      - "--region=us-central1"
    name: gcr.io/cloud-builders/gcloud
user_79505920

With your use case then the best approach would be having different triggers (3 in your use case) that listen to different tags or branches, being each of these specific for the file changes that you want to listen to. At the moment making Cloud Build steps execute when a certain file changes is not available.

user_116410730

If you wanna do it with gcloud CLI

gcloud beta builds triggers create cloud-source-repositories \
--repo=REPO_NAME \
--branch-pattern=BRANCH_PATTERN \ # or --tag-pattern=TAG_PATTERN
--build-config=BUILD_CONFIG_FILE \
--substitutions=_VARIABLE="VALUE"\ 
--included-files "DIRECTORY_NAME/**"

Note:-
--included-files "directory_name/**" will detect all directories and files recursively.
--included-files "directory_name/*" will only look for files in that particular directory.

Example:-

gcloud beta builds triggers create cloud-source-repositories \
--repo=test-repo \
--branch-pattern=master \
--build-config=workflows/cloudbuild.yaml \
--substitutions=_REGION="asia-southeast1"\
--included-files "src/**"
user_84964800
Related