My unit test launch looks like this. As you can see I have exploited CLI options to install a VSIX my CICD has already produced, and then also tried to install ms-vscode-remote.remote-ssh because I want to re-run the tests on a remote workspace.

import * as path from 'path';
import * as fs from 'fs';

import { runTests } from '@vscode/test-electron';

async function main() {
    try {
        // The folder containing the Extension Manifest package.json
        // Passed to `--extensionDevelopmentPath`
        const extensionDevelopmentPath = path.resolve(__dirname, '../../');

        // The path to the extension test runner script
        // Passed to --extensionTestsPath
        const extensionTestsPath = path.resolve(__dirname, './suite/index');

        const vsixName = fs.readdirSync(extensionDevelopmentPath)
            .filter(p => path.extname(p) === ".vsix")
            .sort((a, b) => a < b ? 1 : a > b ? -1 : 0)[0];

        const launchArgsLocal = [
            path.resolve(__dirname, '../../src/test/test-docs'),
            "--install-extension",
            vsixName,
            "--install-extension",
            "ms-vscode-remote.remote-ssh"
        ];
        const SSH_HOST = process.argv[2];
        const SSH_WORKSPACE = process.argv[3];
        const launchArgsRemote = [
            "--folder-uri",
            `vscode-remote://ssh-remote+testuser@${SSH_HOST}${SSH_WORKSPACE}`
        ];

        // Download VS Code, unzip it and run the integration test
        await runTests({ extensionDevelopmentPath, extensionTestsPath, launchArgs: launchArgsLocal });
        await runTests({ extensionDevelopmentPath, extensionTestsPath, launchArgs: launchArgsRemote });
    } catch (err) {
        console.error(err);
        console.error('Failed to run tests');
        process.exit(1);
    }
}

main();

runTests downloads and installs VS Code, and passes through the parameters I supply. For the local file system all the tests pass, so the extension from the VSIX is definitely installed.

But ms-vscode-remote.remote-ssh doesn't seem to be installed - I get this error:

Cannot get canonical URI because no extension is installed to resolve ssh-remote

and then the tests fail because there's no open workspace.

This may be related to the fact that CLI installation of multiple extensions repeats the --install-extension switch. I suspect the switch name is used as a hash key.

What to do? Well, I'm not committed to any particular course of action, just platform independence. If I knew how to do a platform independent headless CLI installation of VS Code:latest in a GitHub Action, that would certainly do the trick. I could then directly use the CLI to install the extensions before the tests, and pass the installation path. Which would also require a unified way to get the path for vs code.

Update 2022-07-20

Having figured out how to do a platform independent headless CLI installation of VS Code:latest in a GitHub Action followed by installation of the required extensions I face new problems.

The test framework options include a path to an existing installation of VS Code. According to the interface documentation, supplying this should cause the test to use the existing installation instead of installing VS Code; this is why I thought the above installation would solve my problems.

However, the option seems to be ignored.

My latest iteration uses an extension dependency on remote-ssh to install it. There's a new problem: how to get the correct version of my extension onto the remote host. By default the remote host uses the marketplace version, which obviously won't be the version we're trying to test.