I have a specific issue with executing jobs if a source image is using in GitLab-ci job any alpine image (test alpine/kaniko, alpine/helm ...).
Infrastructure setup: AWS-fargate-profile(containerd/1.4.1) where is running gitlab-runner,Gitlab-runner-slave are spin up on fargate's ..
Once I will try to use GitLab-runner on EKS (ec2) instances then jobs are properly executed doesn't matter of alpine/debian .. I have tried to replace GitLab-runner-helper image for ubuntu version but still getting same error. EKS-fargate-profile has containerd/1.4.1
I am a little bit confused about what is going wrong or differently on EC2 and fargate
Pipeline job output:
on fargate-gitlab-runner-86c4c7c95f-swbwh XXX
Resolving secrets
00:00
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: gitlab-runner-fg
Using Kubernetes executor with image XXX:8085/alpine/helm:3.2.1 ...
Preparing environment
01:28
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Waiting for pod gitlab-runner-fg/runner-nkzkmtwa-project-386-concurrent-0mbm5j to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Running on runner-nkzkmtwa-project-386-concurrent-0mbm5j via fargate-gitlab-runner-86c4c7c95f-swbwh...
Getting source from Git repository
00:00
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/XXX/0/XXX/fargate-gitlab-runner/.git/
Created fresh repository.
Checking out 497e247d as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:00
Cleaning up file based variables
00:00
ERROR: Job failed (system failure): unable to upgrade connection: container not found ("build")
config.toml
gitlab-version 13.12
listen_address = ":9252"
concurrent = 50
check_interval = 3
log_level = "debug"
[session_server]
session_timeout = 1800
[[runners]]
name = "fargate-gitlab-runner-86c4c7c95f-swbwh"
output_limit = 16384
request_concurrency = 1
url = "https://gitlab.example.com/"
token = "XXX"
executor = "kubernetes"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.kubernetes]
host = ""
bearer_token_overwrite_allowed = false
image = "XX:8085/ubuntu:16.04"
namespace = "gitlab-runner-fg"
namespace_overwrite_allowed = ""
privileged = false
cpu_limit = "1500m"
cpu_request = "1500m"
memory_limit = "4000Mi"
pull_policy = [""]
image_pull_secrets = ["docker-registry-login"]
poll_timeout = 1200
service_account = "infra-fargate-gitlab-runner"
service_account_overwrite_allowed = ""
pod_annotations_overwrite_allowed = ""
helper_image = "XX:8085/gitlab/gitlab-runner-helper:ubuntu-x86_64-c97a73fa"
[runners.kubernetes.affinity]
[runners.kubernetes.pod_security_context]
[runners.kubernetes.volumes]
[runners.kubernetes.dns_config]