Problems with ChromeDriver when running test on GitlabCI

Question

I'm new to Gitlab CI and I'm trying to automatically test my application on commits.

I don't know how this works, I know it uses a docker image and it runs the commands you want.

I picked the node:8 image to start (I am doing an electron project, maybe there is a better image for this)

The thing is, that if I run the command "npm test" on my computer, it runs well and all test pass, but it isn't working on the gitlab ci jobs and I don't know why.

Im developing this on windows, and the docker image uses linux, that could be a problem?

The error always happens with the ChromeDriver. I looked up the folder, and I saw that only had an exe, so I downloaded the linux distribution of the driver and inserted it there. I also execute the driver before running the test (in my computer this isn't needed, it does automatically), and still get the same error.

I'm pretty lost. Any alternative for this to work? Maybe another docker image?

My gitlab-ci.yml:

variables:
  VERSION_ID: '1.0.$CI_PIPELINE_ID'

stages:
  - build

build:
  image: node:8
  stage: build
  artifacts:
    paths:
      - $CI_PROJECT_DIR/dist/*.*
  script:
    - apt-get update
    - apt-get -y install libnss3-dev
    - npm install
    - chmod 0777 ./node_modules/.bin/mocha
    - chmod 0777 ./node_modules/electron-chromedriver/bin/chromedriver
    - ./node_modules/electron-chromedriver/bin/chromedriver&
    - npm test

Error obtained: The testing code (just if this is relevant)

user_3957754 · Answer

What can you do with selenium?

If you electron app is a poll for example, with selenium you could:

Verify that poll app show 10 questions. If you detect minus than 10, throw an error
If your questions are required, you could assure that when a submit button is pressed, a red warning appear next to the each question. Throw an error if submit action does not trigger the warnings and/or form is submitted.

How selenium works

Developer create some tests in a selenium language implementation(java, nodejs, python, etc)
A machine (will call it selenium machine) with a proper browser installation is selected. In this machine at least one browser must be installed and its respective SeleniumDriver. OperaChromiunDriver for opera, FirefoxDriver for firefox, etc
In the selenium machine, you must start a selenium server with proper configurations. This server publish a kind of url with user and password
From your developer laptop you could exec your test is the selenium machine using configurations.
In a developer stage, you could avoid the selenium server, because all required staff already exist on your laptop. But if you are in a enterprise or organization, run tests on your laptop is not possible or recommend. Imagine your self at 2:00 am running tests of all web applications of your work, in your laptop :s. In the other hand imagine your continuous integration server run dozens of tests at 2:00 am and send your a pretty dashboard with the result.

Your questions

Im developing this on windows, and the docker image uses linux, that could be a problem?

Windows is not the best choice to develop. Default browser "wizard" installations will help you at the development but if you promote to the next stage "testing/production" , there isn't any decent cloud provider who attempt to offer WINDOWS for selenium tests or for any technology in the world. Also it is well known the very poor docker support in windows. I advice you linux with https://elementary.io/ which is very cool for programmers like us!

I'm pretty lost. Any alternative for this to work? Maybe another docker image?

You are developing using electron. Electron is a kind of "browser", so Selenium is the best choice to automate tests.

The thing is, that if I run the command "npm test" on my computer, it runs well and all test pass, but it isn't working on the gitlab ci jobs and I don't know why.

Because your test code performs a VISUAL TESTS which require a machine with a visual browser and its respective selenium driver.

what I have to do in order that it works?

www.browserstack.com

As a summary, browserstack provide you a selenium server with a lot of desktop and android browsers. With a few code lines, your tests will run it browserstack cloud and you will have a dashboard with result and videos as proofs of the errors.

Here some alternatives: https://alternative.me/browserstack

Implement you own selenium server

As I said you, you will need a machine with a user interface, browsers, selenium driver, etc

Headless Browser

If you don't have a machine or browserstack is not an option for you, you could research about: Headless Browser

As a summary, a headless browser is a kind of browser implementation in memory, so you could execute your tests in a shell with out user interface (azure, amazon, google, etc).

Anyway you will need the selenium driver an other configurations to achieve that your tests runs in a machine without user interface nor chrome, firefox, opera, etc

Disadvantage of this method is the browser implementation in memory. Some cool funcionalities of your web or javascript could not be recognized. For example: a drag and drop effect. If there isn't a user interface, how this could work :S ?. If just are simple forms , Headless Browser could help you.

Here some options:

https://phantomjs.org/