Not Contains TestCase filter not working in VSTest 16.6.1

Question

According to the official TestCase documentation, you can use a "not contains" (!~) filter in a TestCase filter.

I'm trying to use this filter in an Azure Devops pipeline, but the filter is not supported according to the error I get when running that pipeline:

##[error]Incorrect format for TestCaseFilter Error: Invalid Condition 'FullyQualifiedName!~DSN'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.

This functionality was added in a 16.0 preview version. The pipeline is using VSTest 16.6.1, as evidenced by thes lines in the output:

C:\agent\_work\_tool\VsTest\16.6.1\x64	ools
et451\Common7\IDE\Extensions\TestPlatform\vstest.console.exe "@C:\agent\_work\_temp\xulnpcmgdnx.tmp"
Microsoft (R) Test Execution Command Line Tool Version 16.6.1

However, the "not contains" operator is still not supported.

I have tried:

Using a preview version of VSTest (16.7)
Using "Visual Studio 2019" as test platform version in my VsTest pipeline task (as opposed to directly using a VSTest version)
Using a VSTest version prior to the release that included the "not contains" test filter, which showed the exact same error as expected

I'm at a loss, is there some other kind of dependency that I'm not aware about? Could this be an issue / bug on Microsoft's part? Any help or pointers would be greatly appreciated.

user_10910450 · Answer

This issue could be related to the VSTest version you use. Here're some tests on my side:

Test Execution Command Line Tool Version 15.9.1(Hosted VS2017 agent,Failed):

Test Execution Command Line Tool Version 16.6.0(Hosted Windows2019,Succeeded):

Test Execution Command Line Tool Version 16.7.0-preview-20200519-01(Self-agent,Succeeded)

So i suggest you can run the pipeline with windows 2019 agent first to check if the issue comes from VSTest versions. Then choose correct version.

Not Contains TestCase filter not working in VSTest 16.6.1

1 Answers