Force newman task fail on unsuccessful test results in azure devops pipeline

user_7009784 0
Viewed 1055

I have a simple script in Azure DevOps pipeline YAML executing postman collection tests using newman.

- script: $(Agent.WorkFolder)/node_modules/newman/bin/newman.js run "$(Build.SourcesDirectory)/postman/a.postman_collection.json" -e "$(Build.SourcesDirectory)/postman/b.postman_environment.json" -x -r junit --reporter-junit-export $(Build.ArtifactStagingDirectory)/PostmanResults/JunitResults.xml
  displayName: Run Postman tests

Basically, it works. But this step is always green regardless of tests are passing or not.

Azure Pipeline

How could I force this task/stage fail if there are some unsuccessful tests? Or somehow identify if there are test failures? Does anyone know any clever workaround?

azure-pipelines-yaml npm postman azure-devops
2 Answers

I use the "Newman the cli Companion for Postman" extension instead of a script and it has worked reasonably well for me:

By default, the task will fail your pipeline if there are any test failures.

Here's an example of the task I have configured in my YAML pipelines using it:

- task: carlowaklstedt.NewmanPostman.NewmanPostman.NewmanPostman@4
  displayname: "Run Newman Tests"
  inputs:
    collectionFileSource: "$(System.DefaultWorkingDirectory)"
    contents: "**/path-to-postman-collection.json"
    environment: "**/path-to-environment.json"
    pathToNewman: "(Optional) C:\path\to\newman\installation"
    ignoreRedirect: false
    bail: false
    sslInsecure: false
    reporters: "htmlextra,junit"
    htmlExtraDarkTheme: false
    htmlExtraLogs: false
    htmlExtraTestPaging: false
user_144509540

you have to capture the exit code and throw it as write-error:

Try something like:

if ($exitCode -ne 0)
{
    Write-Output ("[Error] Failing task since return code was {0} while expected 0." -f $exitCode)
    Write-Error "##vso[task.complete result=Failed;]Failed"
}
user_67936370
Related