/bin/bash: line 117: kubectl: command not found gitlab-ci

user_14912334 0
Viewed 2702

I am not able to use kubectl command inside gitlab-ci.yml file.

I have already gone through the steps mentioned in the doc to add an existing cluster in the doc.

Nowhere they have mentioned, How can I use kubectl.

I tried the below configuration.

stages:
  - docker-build
  - deploy

docker-build-master:
  image: docker:latest
  stage: docker-build
  services:
    - docker:dind
  before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  script:
    - docker build --pull -t "$CI_REGISTRY_IMAGE:prod" .
    - docker push "$CI_REGISTRY_IMAGE:prod"
  only:
    - master

deploy-prod:
  stage: deploy
  image: roffe/kubectl
  script:
    - kubectl apply -f scheduler-deployment.yaml
  only:
    - master

But I am getting below error,

Executing "step_script" stage of the job script
00:01
Using docker image sha256:c8d24d490701efec4c8d544978b3e4ecc4855475a221b002a8f9e5e473398805 for roffe/kubectl with digest roffe/kubectl@sha256:ba13f8ffc55c83a7ca98a6e1337689fad8a5df418cb160fa1a741c80f42979bf ...
$ kubectl apply -f scheduler-deployment.yaml
error: unable to recognize "scheduler-deployment.yaml": Get http://localhost:8080/api?timeout=32s: dial tcp [::1]:8080: connect: connection refused
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1

Clearly, it is not able to connect to the cluster, or maybe trying to connect to the cluster inside this roffe/kubectl image container.

When I remove the image, I get this error.

/bin/bash: line 117: kubectl: command not found

I have gone through the whole doc I couldn't find a single example or reference that explains this part.

Please suggest how I can deploy to the existing k8s cluster.

Update

I went through this doc and I am using defined variables in my gitlab-ci.yml to update the context of the kubectl.

But still it doesn't work.

deploy-prod:
  stage: deploy
  image: roffe/kubectl
  script:
    - echo $HOME
    - echo $KUBECONFIG
    - echo $KUBE_URL
    - mkdir -p $HOME/.kube
    - echo -n $KUBECONFIG | base64 -d > $HOME/.kube/config
    - kubectl get pods
  only:
    - master

Error that I get,

$ echo $HOME
/root
$ echo $KUBECONFIG
$ echo $KUBE_URL
$ mkdir -p $HOME/.kube
$ echo -n $KUBECONFIG | base64 -d > $HOME/.kube/config
$ kubectl get pods
The connection to the server localhost:8080 was refused - did you specify the right host or port?
Cleaning up file based variables
00:01
ERROR: Job failed: exit code 1
gitlab gitlab-ci kubernetes docker
2 Answers

To automate deployment with an existing cluster, You need to follow below steps:

1. Add you cluster to gitlab project.

Follow this doc, and add your existing cluster

2. Build your project and push to docker or any registry

stages:
  - docker-build
  - deploy

docker-build-master:
  image: docker:latest
  stage: docker-build
  services:
    - docker:dind
  before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  script:
    - docker build --pull -t "$CI_REGISTRY_IMAGE:prod" .
    - docker push "$CI_REGISTRY_IMAGE:prod"
  only:
    - master

3. Apply your deployment.yml to cluster

To use kubectl inside gitlab-ci.yml, you need an image that has kubectl. The one that you have used will work.

But the kubectl inside the container has no idea about the context of cluster that you added earlier.

Now this is where gitlab's envrionment variable play their role.

The default environment scope is *, which means all jobs, regardless of their environment, use that cluster. Each scope can be used only by a single cluster in a project, and a validation error occurs if otherwise. Also, jobs that don’t have an environment keyword set can’t access any cluster. see here

So, when you added your cluster, it goes by default in scope * and will be passed to every job provided it uses some environment.

Also, When you create a cluster, gitlab create environment variables for that cluster by default, see here.

Important thing to notice is that it also adds an environment variable KUBECONFIG.

In order to access your Kubernetes cluster, kubectl uses a configuration file. The default kubectl configuration file is located at ~/.kube/config and is referred to as the kubeconfig file.

kubeconfig files organize information about clusters, users, namespaces, and authentication mechanisms. The kubectl command uses these files to find the information it needs to choose a cluster and communicate with it.

The loading order follows these rules:

  • If the --kubeconfig flag is set, then only the given file is loaded. The flag may only be set once and no merging takes place.

  • If the $KUBECONFIG environment variable is set, then it is parsed as a list of filesystem paths according to the normal path delimiting rules for your system.

  • Otherwise, the ${HOME}/.kube/config file is used and no merging takes place.

So, kubectl command can use the variable KUBECONFIG to set the context see here.

So, your deployment job will be like below,

deploy-prod:
  stage: deploy
  image: roffe/kubectl
  script:
    - kubectl get pods
    - kubectl get all
    - kubectl get namespaces
    - kubectl apply -f scheduler-deployment.yaml
  environment:
    name: production
    kubernetes:
      namespace: default
  only:
    - master

You can also set a namespace for the job with environment.kubernetes.namespace

user_92667090

this image (roffe/kubectl) doesn't have kubectl package so you can add some package kubectl and configuration to connect your kubernetes cluster.

user_46411650
Related