Publishing test results to Azure (VS Database Project, tSQLt, Azure Pipelines, Docker)

user_15124022 0
Viewed 332

I am trying to fully automate the build, test, and release of a database project using Azure Pipeline.

I already have a Visual Studio solution which consists of three database projects. The first project is the database, which contains the tables, stored procedures, functions, data, etc.. The second project is the tSQLt framework (v 1.0.5873.27393 if anyone is interested). And finally the third project is the tSQLt tests.

My goal here to check the solution into source control, and the pipeline will automatically build the solution, deploy the dacpacs to a build server (docker in this case), run the tSQLt tests, and publish the results back to the pipeline.

My pipeline works like this.

  • Building the visual studio solution
  • Publish the Artifacts
  • Setup a docker container running Ubuntu & SQL Server
  • Install SQLPackage
  • Deploy the dacpacs to the SQL instance
  • Run the tSQLt tests
  • Publish the test results

Everything up to publishing the results is working, but on this step I got the following error:

[warning]Failed to read /home/vsts/work/1/Results.xml. Error : Data at the root level is invalid. Line 1, position 1.

I added another step in the pipeline to display the content of the Results.xml file. It appears like this:

XML_F52E2B61-18A1-11d1-B105-00805F49916B                                                                                                                                                                                                                        
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
<testsuites><testsuite id="1" name="MyNewTestClassOne" tests="1" errors="0" failures="0" timestamp="2021-02-01T10:40:31" time="0.000" hostname="f6a05d4a3932" package="tSQLt"><properties/><testcase classname="MyNewTestClassOne" name="TestNumberOne" time="0.

I'm not sure if the column name and dashes should be in the file, but I'm guessing not. I added another step in to remove them, just leaving me with the XML. But this then gave me a different error to deal with:

##[warning]Failed to read /home/vsts/work/1/Results.xml. Error : There is an unclosed literal string. Line 2, position 1.

This one is a little obvious to spot, because as you'll see above, the XML is incomplete.

Here is the part of my pipeline which runs the tSQLt tests and outs the results to Results.xml

- script: |
    sqlcmd -S 127.0.0.1,1433 -U SA -P Password.1! -d StagingDB -Q 'EXEC tSQLt.RunAll;'
  displayName: 'tSQLt - Run All Tests'
- script: |
    cd $(Pipeline.Workspace)
    sqlcmd -S 127.0.0.1,1433 -U SA -P Password.1! -d StagingDB -Q 'SET NOCOUNT ON; EXEC tSQLt.XmlResultFormatter;' -o 'tSQLt_Results.xml'
  displayName: 'tSQLt - Output Results'

I've research so many blogs and articles on this, and most people are doing the same. Some people use PowerShell instead of sqlcmd, but given I'm using a Ubuntu machine this isn't an option here.

I am all out of options, so I am looking for a little help on this.

tsqlt azure-pipelines docker azure tsql
1 Answers

You are dealing with 2 problems here. There is noise in your result set, that is not xml and your xml result is truncated after 256 characters. I can help you with both.

What I am doing is basically this:

/opt/mssql-tools/bin/sqlcmd \
    -S "localhost, 31114" -U sa \
    -P "password" \
    -d dbname \
    -y0 \
    -Q "BEGIN TRY EXEC tSQLt.RunAll END TRY BEGIN CATCH END CATCH; EXEC tSQLt.XmlResultFormatter" \
        | grep -w "<testsuites>" \
        | tee "resultfile.xml"

Few things to note:

  • y0 important. This sets the length of the xml result set to unlimited, up from 256.
  • grep with a regular expression - make sure you only get the xml and not the noise around it.

If you want to run only a subset of your tests, you need to make amendments to the SQL query being passed in, but other than that, this is a catch it all "oneliner" to run all tests and get the results in xml format, readable by Azure DevOps

user_12496410
Related