CodeBuild doesn't report failed tests

Viewed 2259

I am using CodeBuild to build my NPM project. I have a report group specific in my buildspec and I am running unit tests using Jest and the npm test command.

When all the tests pass, they are reported to the CodeBuild report group successfully. However if one of the tests fails I get the error:

[Container] 2020/08/24 01:41:18 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: npm test -- --silent. Reason: exit status 1`

and the build stops (which is fine in general) but the failed test is not reported so the test pass rate remains at 100% and means I don't see test trends or other details.

The relavent parts of my buildspec.yml:

version: 0.2
phases:
  install:
    commands:  
      - npm install
  pre_build:
    commands:
      - npm test -- --silent    
reports:
  jest_reports:
    files:
      - 'test-results.xml'
    file-format: JunitXml
    base-directory: "reports/results"

The Jest config in my package.json:

"jest": {
    "reporters": [
        "default",
        [
            "jest-junit",
            {
                "outputDirectory": "reports/results",
                "outputName": "test-results.xml"
            }
        ]
    ]
}

The final part of the CodeBuild log:

Test Suites: 1 failed, 8 passed, 9 total
Tests:       2 failed, 57 passed, 59 total
Snapshots:   0 total
Time:        3.034s
npm ERR! Test failed.  See above for more details.

[Container] 2020/08/24 01:41:18 Command did not exit successfully npm test -- --silent exit status 1
[Container] 2020/08/24 01:41:18 Phase complete: PRE_BUILD State: FAILED
[Container] 2020/08/24 01:41:18 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: npm test -- --silent. Reason: exit status 1

How can I setup CodeBuild to report failed tests, but still prevent it from continuing the build if a test does fail?

1 Answers

In researching this, all of the examples that I see that use CodeBuild test reports, the tests are run as part of the build phase.

Since your tests are in the pre_build phase, if the pre_build fails, it won't proceed to the upload_artifacts phase and will just skip to finalizing. See the CodeBuild phase transitions diagram for more information on this.

I'd recommend following the patterns of the user guides and putting your tests in the build phase.

User guide references:

Related