Google Cloud Build cannot run NX affected:apps because it cannot reference the master~1 commit

user_11662073 0
Viewed 2406

I am trying to set up a monorepo complete CI/CD pipeline developed with nx.dev where I build and deploy only the apps and services that have changed in the commit.

My cloud build is linked to my github repo and when a change is pushed it kicks off a build. First npm install and then build the changed apps.

As per nrwls docs on nx https://nx.dev/guides/monorepo-affected#ci they say to use

npm run affected:build -- --base=origin/master~1 --head=origin/master

This will compare the current commit with the previous commit to find out which services or apps to build.

I have tried using this but am getting this error when running in cloud build 

Step #1: fatal: Not a valid object name master~1
Step #1: Command failed: git merge-base master~1 master
Step #1: fatal: Not a valid object name master~1

When building this locally with cloud-build-local it works fine and successfully figures out which services to build.

I think the reason that it is failing is because when cloud build checks out the git repository it only checks out the commit and none of the previous commit information. Therefore it cannot reference the previous commit.

Is there any way around this or am I missing something?

Thanks!

nrwl nrwl-nx google-cloud-build git
4 Answers

You might be able to do it with an expression.

npm run affected:build -- --base=$(git rev-parse HEAD~1) --head=origin/master

This is how I do it with github actions

name: Test develop and feature branches

on:
  push:
    branches:
      - develop
      - "feature/*"

jobs:
  test:
    name: Test
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v1

      - name: Use node.js 12
        uses: actions/setup-node@v1
        with:
          node-version: 12

      - name: Cache Yarn
        uses: actions/cache@v1
        with:
          path: node_modules
          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.OS }}-yarn-${{ env.cache-name }}-
            ${{ runner.os }}-yarn-

      - name: Yarn install
        run: yarn install --frozen-lockfile --non-interactive

      - name: Retreive last test sha
        id: last-test-sha
        run: |
          if [[ $GITHUB_BASE_REF ]]
          then
            echo "::set-output name=sha::remotes/origin/$GITHUB_BASE_REF"
          else
            echo "::set-output name=sha::$(git rev-parse HEAD~1)"
          fi

      - name: Run affected tests
        run: yarn affected:test --ci --runInBand --base=${{ steps.last-test-sha.outputs.sha }} --head=${GITHUB_SHA}
        env:
          CI: "true"
          TZ: "utc"
user_712450

I was facing the same issue, the reason is affected needs deep checkout as it diffs against master. So following changes in GHA would resolve it:

steps:
   - uses: actions/checkout@v2
     with:
       fetch-depth: 0
user_56440900

Cloud Build does not fetch neither branches nor history.

Including the repository history in a build

Cloud Build does not check out any other branches or history. This is done for efficiency, so that builds don't have to wait to fetch the whole repository and history just to build a single commit.

So they propose to fetch the history during the build. For example, it's a hacky example how you could add .git folder to your build workspace, so NX could calculate affected projects.

  - id: Get the .git directory
    name: git
    args:
      - bash
      - -c
      - |-
        # get the .git directory only
        git clone --branch $BRANCH_NAME git@github.com:<USER_NAME>/$REPO_NAME /tmp/repo ;
        # move it to the workspace
        mv /tmp/repo/.git .

If you have a private repository, there is an article describing auth process.

user_17596140

As another has mentioned, other branches are not fetched.

In addition to the branch that has triggered your test/build/deploy, you have to fetch a commit or branch. In our case we use a branch that represents the last successful build.

Here's an example of our script (in Codeship). Currently using nx v7. Works well for us so far. 

## fetch the last successfully deployed branch to determine affected build
git fetch --no-tags --prune --depth=5 origin lastdeploy_production:refs/remotes/origin/lastdeploy_production
nx affected:test --base=remotes/origin/lastdeploy_production --head=HEAD
nx affected:build --prod --parallel --maxParallel 8 --base=remotes/origin/lastdeploy_production --head=HEAD
# ...deployment commands (removed)...
## after a successful deployment we force push to this branch for comparing affected
git push origin HEAD:lastdeploy_production -f
user_135936810
Related