Kubernetes add Toleration from CLI

Question

I'm using the Oracle Cloud Infrastructure with Kubernetes and Docker. I've got the following pod:

$ kubectl describe pod $podname -n $namespace
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  19m   default-scheduler  0/1 nodes are available: 1 node(s) had taint {nvidia.com/gpu: }, that the pod didn't tolerate.
  Warning  FailedScheduling  18m   default-scheduler  0/1 nodes are available: 1 node(s) had taint {nvidia.com/gpu: }, that the pod didn't tolerate.

I want to add a toleration to this pod - is there a command to do so, without creating the pod config yaml file (as this pod is created by some other systems that I don't want to edit. I just want to add the toleration to resolve this issue.

Thanks.

====================

gpu-config.yaml

apiVersion: v1 # What version of the Kubernetes API to use
kind: Pod      # What kind of object you want to create
metadata:      # Data that helps uniquely identify the object, including a name, string, UID and optional namespace
  name: nvidia-gpu-workload
spec:          # What state you desire for the object, differs for every type of Kubernetes object.
  restartPolicy: OnFailure
  containers:
    - name: cuda-vector-add
      image: k8s.gcr/io/cuda-vector-add:v0.1
      resources:
        limits:
          nvidia.com/gpu: 1
  tolerations:
  - key: "nvidia.com/gpu"
    operator: "Equal"
    effect: "NoSchedule"

# Update command
$ kubectl create -f ./gpu-config.yaml
# All this seems to do is create a pod by the name of nvidia-gpu-workload-v2, and it doesn't add these configurations to the pod that I require.

Just to note that this issue is occurring on a pod called hook-image-awaiter-5tq5 and I don't think I should re-create that pod with a different config as it seems to be configured by part of the system.

Kubernetes add Toleration from CLI

0 Answers