how to create test report files for bitbucket pipelines?

hanshenrik 0
Viewed 75

How does one create test result files for bitbucket pipelines? My bitbucket bitbucket-pipelines.yml contains:

options:
  docker: true

pipelines:
  default:
    - step:
        name: test bitbucket pipelines stuff..
        script: # Modify the commands below to build your repository.
          - /bin/bash -c 'mkdir test-results; echo Error Hello, World >> test-results/test1.txt; find'

and when running this pipeline i get

/bash -c 'mkdir test-results; echo Error Hello, World >> test-results/test1.txt; find'
<1s
+ /bin/bash -c 'mkdir test-results; echo Error Hello, World >> test-results/test1.txt; find'
.
(... censored/irrelevant stuff here)
./test-results
./test-results/test1.txt

then i get the "build teardown" saying it can't find test-results/test1.txt:

Build teardown
<1s
Searching for test report files in directories named
 [test-results, failsafe-reports, test-reports, TestResults, surefire-reports] down to a depth of 4
Finished scanning for test reports. Found 0 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.

i am surprised that it failed to find the ./test-results/test1.txt file.. hence the question.

bitbucket-pipelines bitbucket
2 Answers

per https://support.atlassian.com/bitbucket-cloud/docs/test-reporting-in-pipelines/ seems it has to be XML files.. also it needs to be in a "j-unit xml format" ? an example of which can be found here https://www.ibm.com/docs/en/developer-for-zos/9.1.1?topic=formats-junit-xml-format

.. so try changing bitbucket-pipelines.yml to

options:
  docker: true

pipelines:
  default:
    - step:
        name: test bitbucket pipelines stuff..
        script: # Modify the commands below to build your repository.
          - export IMAGE_NAME2=easyad/easyad_nginx:$BITBUCKET_COMMIT
          - /bin/bash bitbucket_pipeline_tests.sh

and in bitbucket_pipeline_tests.sh add

#!/bin/bash
mkdir test-results;
echo '<?xml version="1.0" encoding="UTF-8" ?> 
   <testsuites id="20140612_170519" name="New_configuration (14/06/12 17:05:19)" tests="225" failures="1262" time="0.001">
      <testsuite id="codereview.cobol.analysisProvider" name="COBOL Code Review" tests="45" failures="17" time="0.001">
         <testcase id="codereview.cobol.rules.ProgramIdRule" name="Use a program name that matches the source file name" time="0.001">
            <failure message="PROGRAM.cbl:2 Use a program name that matches the source file name" type="WARNING">
WARNING: Use a program name that matches the source file name
Category: COBOL Code Review – Naming Conventions
File: /project/PROGRAM.cbl
Line: 2
      </failure>
    </testcase>
  </testsuite>
</testsuites>' >>./test-results/test1.xml

then the pipeline run should say 17 / 45 tests failed, as indicated by the sample XML above...

hanshenrik0

Usually, each language/framework has some kind of utility to automatically produce such files as an outcome of a test suite run.

E.g. in Python you could simply run

pytest --junitxml=test-results/pytest.xml

See https://docs.pytest.org/en/latest/how-to/output.html#creating-junitxml-format-files

Manually crafting the xml youself feels brittle and tedious. Better find whatever library/option is available for your language/framework.

N1ngu0
Related