I'm working on a Flutter project and need to run integration tests on a Gitlab CI pipeline.
The tests work fine locally, launching chromedriver -port=4444 and then flutter drive --driver=integration_test/test_driver/integration_test.dart --target=integration_test/integration/app_test.dart.
However, I cannot get them to run in CI. My yaml file looks like this :
...
test_ui_integration:
needs:
- build_deploy_user_service
- populate_db
tags:
- docker
stage: test_ui_integration
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
when: always
before_script:
- chromedriver --port=4444 & # launching the chromedriver
- sleep 10
script:
- cd $CI_PROJECT_DIR/ui
- npm ${NODE_INSTALL_OPTIONS:-i}
- npm run build
- flutter drive --driver=integration_test/test_driver/integration_test.dart --target=integration_test/integration/app_test.dart -d chrome --web-run-headless --headless
# - flutter drive --driver=integration_test/test_driver/integration_test.dart --target=integration_test/integration/app_test.dart -d web-server --headless
# - flutter drive --driver=integration_test/test_driver/integration_test.dart --target=integration_test/integration/app_test.dart -d web-server --release --web-run-headless --headless
timeout: 40 minutes
(note that the pipeline is using a docker img with chromedriver and Flutter installed)
I have tried all kinds of different combination for the flags, as seen here too. However, none of it seems to work: Even locally, I cannot get the test to run headless-ly (without a browser).
The CI step usually gets stuck on something like this, until it times out :
=== running with flags -d web-server --web-run-headless --headless ===
Waiting for connection from debug service on Web Server... 37.1s
integration_test/integration/app_test.dart is being served at http://localhost:44415
The web-server device requires the Dart Debug Chrome extension for debugging. Consider using the Chrome or Edge devices for an improved development workflow.
...
=== running with flags -d chrome --web-run-headless --headless ===
Waiting for connection from debug service on Chrome... 37.5s
[CHROME]:[1391:1391:0609/090100.591700:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
Failed to launch browser after 3 tries. Command used to launch it: google-chrome --user-data-dir=/tmp/flutter_tools.RDVJEL/flutter_tools_chrome_device.KRPHNF --remote-debugging-port=41647 --disable-background-timer-throttling --disable-extensions --disable-popup-blocking --bwsi --no-first-run --no-default-browser-check --disable-default-apps --disable-translate http://localhost:35691
I've also noticed that you can run integration tests locally with flutter run path/to/my/test/dart, which is something that the official docs never mention. Is there a difference between the two?
Should I be trying to run the CI tests with a mobile emulator instead?
Any help much appreciated!
EDIT: As requested by @MarkosTh09, here is the dockerfile and relevant CI workflow :
FROM fedora:35
# Add basic tools
RUN dnf -y install git make gcc gcc-c++ wget unzip zip jq diffutils \
&& dnf clean all
# Requirement of openapi generator
RUN dnf install -y java \
&& dnf clean all
# Install python
RUN dnf install -y python3 \
&& dnf clean all
# Install chromedriver
RUN wget https://chromedriver.storage.googleapis.com/99.0.4844.35/chromedriver_linux64.zip \
&& unzip chromedriver_linux64.zip \
&& cp chromedriver /usr/bin/chromedriver \
&& chown root /usr/bin/chromedriver \
&& chmod +x /usr/bin/chromedriver \
&& chmod 755 /usr/bin/chromedriver
# Install chrome
RUN wget https://dl.google.com/linux/chrome/rpm/stable/x86_64/google-chrome-stable-99.0.4844.84-1.x86_64.rpm \
&& yum install ./google-chrome-*.rpm -y
# Install AWS CLI V2
RUN curl -s https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip > awscliv2.zip \
&& unzip awscliv2.zip \
&& ./aws/install \
&& rm -rf aws awscliv2.zip
# Install and configure nvm for npm
# nvm environment variables
ENV NVM_DIR /opt/nvm
RUN mkdir -p $NVM_DIR
ENV NODE_VERSION 16.14.0
# install nvm
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
# install node and npm
RUN source $NVM_DIR/nvm.sh \
&& nvm install $NODE_VERSION \
&& nvm alias default $NODE_VERSION \
&& nvm use default
# add node and npm to path so the commands are available
ENV NODE_PATH $NVM_DIR/versions/node/v$NODE_VERSION/lib/node_modules
ENV NVM_BIN $NVM_DIR/versions/node/v$NODE_VERSION/bin
ENV NVM_INC $NVM_DIR/versions/node/v$NODE_VERSION/include/node
ENV PATH $NVM_BIN:$PATH
# Install serverless
RUN npm install -g serverless@3.2.0
# Install Flutter
RUN curl -s https://storage.googleapis.com/flutter_infra_release/releases/stable/linux/flutter_linux_3.0.0-stable.tar.xz > flutter.tar.xz \
&& tar xf flutter.tar.xz -C /opt/ \
&& rm flutter.tar.xz
ENV PATH $PATH:/opt/flutter/bin
RUN git config --global --add safe.directory /opt/flutter
CMD [ "/bin/bash" ]
ENTRYPOINT [ "/bin/bash", "-c" ]
include:
- '/ci/ci-build-deploy.yml'
- '/ci/ci-tests.yml'
- '/ci/ci-undeploy.yml'
image: registry.gitlab.com/project/project/build/project-build-env:4
variables:
PROJECT_API_URL: https://gitlab.com/api/v4/projects/12345678
NODE_INSTALL_OPTIONS: ci --cache $CI_PROJECT_DIR/.npm --prefer-offline
cache:
key: PROJECT_NODE_CACHE
paths:
- .npm/
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
variables:
AWS_STAGE: "ci${CI_PIPELINE_ID}"
STAGE_TYPE: "ci"
- if: $STAGE_TYPE == "staging" && $AWS_STAGE == "staging"
variables:
HOSTED_ZONE_NAME: "staging.project.io"
FRONT_DOMAIN: "staging.project.io"
- if: $STAGE_TYPE == "prod" && $AWS_STAGE == "prod"
variables:
HOSTED_ZONE_NAME: "project.io"
FRONT_DOMAIN: "project.io"
- when: always
stages:
- build_deploy_user_service
- build_deploy_shop_service
- build_deploy_brand_service
- build_deploy_product_service
- build_deploy_web_ui
- build_deploy_reverseproxy
- test_shop_ws
- test_brand_ws
- test_product_ws
- delete_test_objects
- populate_db
- test_ui
- populate_cognito_test_user
- test_ui_integration
- triggerundeploy
- undeploy