How to put self-signed certificate to each node of GKE cluster?

user_3319341 0
Viewed 2113

I have a kubernetes cluster in GKE. Inside the cluster there is an private docker registry service. A certificate for this service is generated inside a docker image by running:

openssl req -x509 -newkey rsa:4096 -days 365 -nodes -sha256 -keyout /certs/tls.key -out /certs/tls.crt -subj "/CN=registry-proxy"

When any pod that uses an image from this private registry tries to pull the image I get an error:

x509: certificate signed by unknown authority

Is there any way to put the self signed certificate to all GKE nodes in the cluster to resolve the problem?

UPDATE

I put the CA certificate to each GKE node as @ArmandoCuevas recommended in his comment, but it doesn't help, still getting the error x509: certificate signed by unknown authority. What could cause it? How docker images are pulled into pods?

google-kubernetes-engine kubernetes docker google-cloud-platform
1 Answers

TL;DR: Almost all modifications you need to perform to nodes in GKE, like adding trusted root certificates to the server, can be done using Daemonsets.

There is an amazing guide that the user Sam Stoelinga created about how to perform what you are looking to do. The link can be found here.

As a summary, the way Sam propose how to perform this changes is by distributing the cert in each of the nodes by using a Daemonsets. Since the Daemonsets guarantees that there is 1 pod on each of the nodes always, the POD will be in charge of adding your certificate to the node so you can pull your images from the private registry.

Normally adding the node by your own will not work since if GKE needs to recreate the node you change will be lost. This approach of using DS guarantees that even if the node is recreated, since the Daemonset will schedule one of this "overhaul pod" in the node, you will always going to have the cert in place.

The steps that Sam proposed are very simple:

  1. Create an image that with the commands needed to distribute the certificate. This step may be different if you are using Ubuntu nodes or COS nodes. If you are using COS nodes, the commands that your pod needs to run if you are using COS are perfectly outlined by SAM:
cp /myCA.pem /mnt/etc/ssl/certs
nsenter --target 1 --mount update-ca-certificates
nsenter --target 1 --mount systemctl restart docker

If you are running Ubuntu nodes, the commands are outlined in several posts in Ask Ubuntu like this one.

  1. Move the image to a container registry that your nodes currently have access like GCR.

  2. Deploy the DS using the image that adds the cert as privileged with the NET_ADMIN capability (needed to perform this operation) and mount host's "/etc" folder inside the POD. Sam added an example of doing this that may help, but you can use your own definition. If you face problems while trying to deploy a privileged pod, may worth taking a look to GKE documentation about Using PodSecurityPolicies

user_120105910
Related