Cypress Parallelisation Without Cypress Dashboard

Look at this free solution https://github.com/agoldis/sorry-cypress I use only director service in docker-compose to make tests run parallel , as a simple example:

version: '3.7'
networks:
  default:
    external:
      name: bridge

services:
  cypress:
    container_name: cypress
    build:
      context: ../
      dockerfile: ./docker/cy.Dockerfile
    links:
      - director
    ports:
      - '5555:5555'
    network_mode: bridge

  cypress2:
    container_name: cypress2
    build:
      context: ../
      dockerfile: ./docker/cy.Dockerfile
    links:
      - director
    ports:
      - '5556:5556'
    network_mode: bridge

  mongo:
    image: mongo:4.0
    network_mode: bridge
    ports:
      - 27017:27017

  director:
    image: agoldis/sorry-cypress-director:latest
    environment:
      MONGODB_URI: "mongodb://mongo:27017"
    network_mode: bridge
    ports:
      - 1234:1234
    depends_on:
      - mongo

you can try this cypress-parallel-specs-locally

All these answers seem fine, but I wanted something very simple to use with GitLab that does not requiring additional external services. I came up with this very simple wrapper around Cypress:

const path = require('path');
const fs = require('fs');
const childProcess = require('child_process');
const glob = require('glob-promise');

async function getIntegrationDirectory() {
    let integrationFolder = 'cypress/integration';
    let cypressJson = path.join(process.cwd(), 'cypress.json');
    try {
        await fs.promises.access(cypressJson, fs.constants.F_OK);
        let cypressConfig = require(cypressJson);
        integrationFolder = cypressConfig.integrationFolder ?? integrationFolder;
    }
    catch (err) {
        // Ignore if file does not exist
    }
    return integrationFolder;
}

async function main() {
    let nodeIndex = parseInt(process.env.CI_NODE_INDEX ?? 1, 10);
    let totalNodes = parseInt(process.env.CI_NODE_TOTAL ?? 1, 10);
    let integrationFolder = await getIntegrationDirectory();
    let specs = await glob(integrationFolder + '/**/*.js');
    let start = (nodeIndex - 1) * specs.length / totalNodes | 0;
    let end = nodeIndex * specs.length / totalNodes | 0;
    let specsToRun = specs.slice(start, end);
    let prefix = `Parallel Cypress: Worker ${nodeIndex} of ${totalNodes}`;
    if (!specsToRun.length) {
        console.log(`${prefix}, no specs to run, ${specs.length} total`);
        return;
    }
    console.log(`${prefix}, running specs ${start + 1} to ${end} of ${specs.length} total`);
    let args = process.argv.slice(2)
        .concat(['--spec', specsToRun.join(',')]);
    let child = childProcess.spawn('cypress', args, {stdio: 'inherit', shell: true});
    await new Promise((resolve, reject) => {
        child.on('exit', exitCode => {
            if (exitCode) {
                reject(new Error(`Child process exited with code ${exitCode}`));
            }
            resolve();
        });
    });
}

main().catch(err => {
    console.error(err);
    process.exit(1);
});

It requires glob-promise and glob to be installed.

It integrates nicely with GitLab's parallel keyword and would work with other CI runners with minimal modification.

To use it, simply add parallel: 4 to the GitLab CI job, and then call node parallelCypress.js <some options>. It will then scan for tests and split them up to the available parallel jobs, then invoke cypress with the --spec option, so that each job only runs a part of the tests.

There's a lot of room for improvements though, especially since not all spec files contain the same number of tests and not all tests have similar runtimes.

if you use Github Actions, I have created this workflow that runs tests in parallel, up to 15 spec files per parallel run, so it will automatically scale when adding new tests. It may help you or others. This also has the app run in https mode, but you can remove those lines if not needed.

https://pastebin.com/ubx8BdUn

# This is a basic workflow to help you get started with Actions
name: Project
 
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
  push:
    branches: [master, preprod, staging]
  pull_request:
    branches: [master, preprod, staging]
 
jobs:
  setup:
    runs-on: ubuntu-latest
    outputs:
      # will contain a json string with an array of n elements, each being a string of spec files delimited by ,
      test-chunks: ${{ steps['set-test-chunks'].outputs['test-chunks'] }}
      # json string with ids to use in the next job matrix depending on how many elements are in the above array, eg: [0,1]
      test-chunk-ids: ${{ steps['set-test-chunk-ids'].outputs['test-chunk-ids'] }}
    steps:
      - uses: actions/checkout@v2
      - id: set-test-chunks
        name: Set Chunks
        # get all spec files from the integration directory, group them to be at most 15 at a time and transform them to json
        run: echo "::set-output name=test-chunks::$(find cypress/integration -type f -name "*.spec.js" | xargs -n15 | tr ' ' ',' | jq -R . | jq -s -cM .)"
      - id: set-test-chunk-ids
        name: Set Chunk IDs
        # get the number of elements from the above array as an array of indexes
        run: echo "::set-output name=test-chunk-ids::$(echo $CHUNKS | jq -cM 'to_entries | map(.key)')"
        env:
          CHUNKS: ${{ steps['set-test-chunks'].outputs['test-chunks'] }}
 
  tests:
    needs:
      - setup
    runs-on: ubuntu-latest
    container:
      # use cypress image, since just using node 12 doesn't work currently for some reason, gives node-sass error
      image: cypress/browsers:node12.13.0-chrome78-ff70
      options: "--ipc=host" # fix for a cypress bug
    name: test (chunk ${{ matrix.chunk }})
    strategy:
      matrix:
        # will be for eg chunk: [0,1]
        chunk: ${{ fromJson(needs.setup.outputs['test-chunk-ids']) }}
    steps:
      - name: Checkout
        uses: actions/checkout@v2
 
      - name: Add domain to hosts file
        run: echo "127.0.0.1 your.domain" | tee -a /etc/hosts
 
      # cache cypress and node_modules for faster operation
      -  uses: actions/cache@v2
         with:
           path: '~/.cache/Cypress'
           key: ${{ runner.os }}-cypress-${{ hashFiles('**/yarn.lock') }}
 
      - uses: actions/cache@v2
        with:
          path: '**/node_modules'
          key: ${{ runner.os }}-modules-docker-${{ hashFiles('**/yarn.lock') }}
 
      # in case cache is not valid, install the dependencies
      - run: yarn --frozen-lockfile
      - run: yarn run cypress install
 
      # run the frontend server in background and wait for it to be available
      - run: PORT=443 HTTPS=true yarn ci-start &
      - run: npx wait-on https://your.domain --timeout 180000
 
      # the cypress docker doesn't contain jq, and we need it for easier parsing of json array string.
      # This could be improved in the future, but only adds ~2s to the build time
      - run: apt-get install jq -y
 
      - name: Run Cypress
        run: SPECS=$(echo $CHUNKS | jq -cMr '.[${{ matrix.chunk }}] | @text') && yarn cypress:ci --spec $SPECS
        env:
          NODE_TLS_REJECT_UNAUTHORIZED: 0
          CHUNKS: ${{ needs.setup.outputs['test-chunks'] }}
 
  testsall:
    if: ${{ always() }}
    runs-on: ubuntu-latest
    name: Tests All
    needs: tests
    steps:
      - name: Check tests matrix status
        if: ${{ needs.tests.result != 'success' }}
        run: exit 1

I've created a Github Action inspired by the answer of Bielik that allows, among other features, to specify the number of test runners.

Execution consists of two parts: First, a prepare/install step that splits the tests into specs (best combined with what Cypress Dashbaord needs):

...
- uses: tgamauf/cypress-parallel@v1

This provides two output variables integration-tests and if configured component-tests, which can then be used in the test step:

    strategy:
      fail-fast: false
      matrix:
        spec: ${{ fromJson(needs.install.outputs.integration-tests) }}
...
      - name: Execute tests
        uses: cypress-io/github-action@v2
        with:
          # We have already installed all dependencies above
          install: false
          # Use the spec provided to this worker to execute the test 
          spec: ${{ matrix.spec }}

Try adding Cypress action in Buddy CI/CD. That way you will build and test your app on every push (and they deploy wherever you want). I think Buddy should help you with that issue.

enter image description here

If using GitLab, you can always manually split the jobs through the .gitlab-ci.yml file, like this:

 1-job:
   stage: acceptance-test
   script:
     - npm install
     - npm i -g wait-on
     - wait-on -t 60000 -i 5000 http://yourbuild
     - npm run cypress -- --config baseUrl=http://yourbuild --spec ./**/yourspec1

 2-job:
   stage: acceptance-test
   script:
     - npm install
     - npm i -g wait-on
     - wait-on -t 60000 -i 5000 http://yourbuild
     - npm run cypress -- --config baseUrl=http://yourbuild --spec ./**/yourspec2

I have created the orchestrator tool to be able to do that. It allows you to deploy any number of cypress docker containers and gonna split all specs across them and at the end, it's gonna generate a beautiful HTML report.

It is open-source, free to use, and You could use it with Jenkins, TravisCI, github actions, or any other CI.

Using GitHub Actions I have devised a solution that may be an overkill, but does not require Cypress Dashboard and still allows users to run their e2e tests in parallel. Solution is in private project so I cannot share a link, but I can give you simplified code snippets, that will hopefully be enough to get you started.

The idea is to run each spec on separate machine using GitHub Actions. You run this script somewhere in your install job (remember to output it globally):

import * as core from '@actions/core';
import * as glob from 'glob';

export function setE2E() {
  const specs = getAllSpecFiles();

  core.info(JSON.stringify({ specs }, null, 2));

  core.setOutput('e2e-files', specs);
}

function getAllSpecFiles() {
  let projectRoot; // fill
  let pattern; // fill eg. "./src/integration/**/*.spec.ts"

  return glob.sync(pattern, {
    cwd: projectRoot,
    dot: true,
    silent: true,
    follow: true,
    nodir: true,
  });
}

This will get all spec files from your cypress project. Then in your e2e job something like:

    strategy:
      fail-fast: false
      matrix:
        spec: ${{ fromJson(needs.install.outputs.e2e-files) }}
...
    - name: Run Spec
      run: npx cypress run --spec ${{ matrix.spec }}

This will spawn separate machine for each spec in your cypress project. As I said, a little bit an overkill, but it is quite simple to configure and gets the job done.

Testery.io is a cloud based testing platform that supports running Cypress tests in parallel. You can sign up for a free plan to run up to 5 tests in parallel, integrate the execution into your ci/cd system, and view the results on the platform. You can also run 15-30 tests in parallel if you choose a paid plan: https://testery.io/pricing.