Starting Minikube with Docker driver and bind it to host network

user_5488750 0
Viewed 6607

I was wondering if it was possible to bind my minikube network to my host network.

I tried:

minikube start --memory=10000 --cpus=4 --vm-driver=docker --kubernetes-version=v1.19.6 --mount --mount-string="/usr/local/citizennet/db:/usr/local/citizennet/db" --network="host"

But I'm getting the following error:

ā—  Unable to create dedicated network, this might result in cluster IP change after restart: un-retryable: create network host 192.168.49.0/24: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true host: exit status 1
stdout:

stderr:
Error response from daemon: operation is not permitted on predefined host network

I was able to do that by using haproxy but I would like to know if there is a cleaner way of doing that. My minikube is hosted on an EC2 instance and I would like to forward everything to my minikube directly. Or at least the HTTP/HTTPS requests.

Thanks!

minikube kubernetes
1 Answers

I haven't found a way to expose the minikube instance with --driver=docker to the host network (apart from $ kubectl port-forward svc/svc-name --address=0.0.0.0 local_port:pod_port ran on the host).

It produces the same error as original poster is experiencing:

Error response from daemon: operation is not permitted on predefined host network

Acknowledging following comment:

the problem is that I want to use the ingress addon and this addon is not compatible anymore with --driver=none.

Instead of using --driver=docker which will place all of the resources in the Docker container, you can opt for a --driver=none which will provision all of your resources directly on the VM. You will be able to directly query the resources from other network devices.

For now minikube version v1.17.1 does not allow to use the ingress addon with --driver=none but I found a way it could be provisioned. I've included this example on the end of this answer. Please treat this as a workaround.

This issue (inability to use ingress addon on --driver=none) is already addressed on github:

Talking from the perspective of exposing minikube:

As it's intended for accessing from external sources, I do recommend trying out other solutions that will subjectively speaking have easier time exposing your workloads to the external sources. There are many available tools that spawn Kubernetes clusters and you can look which suits your needs the most. Some of them are:

Deploying nginx-ingress with minikube --driver=none

As stated previously, please treat it as a workaround.

A side note!

Take a look on how your NGINX Ingress controller is configured with minikube addons enable ingress as it will be pretty much mimicked in this example.

Steps:

  • Download the nginx-ingress YAML manifest:
    • Modify the Deployment in the manifest
    • Delete the Service from manifest
  • Apply and check

Download the nginx-ingress YAML manifest

You can use following manifest:

Modify the Deployment in the manifest

As I said previously, what is happening when you run minikube addons enable ingress could prove useful. The resources deployed have some clues on how you need to modify it.

  • Add the hostPort for HTTP and HTTPS communication:
          ports:
            - name: http
              hostPort: 80 # <-- IMPORTANT, ADD THIS
              containerPort: 80
              protocol: TCP
            - name: https
              hostPort: 443 # <-- IMPORTANT, ADD THIS
              containerPort: 443
              protocol: TCP
            - name: webhook
              containerPort: 8443
              protocol: TCP
  • Delete the --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller:
          args:
            - /nginx-ingress-controller
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller # <-- DELETE THIS
            - --election-id=ingress-controller-leader
            - --ingress-class=nginx
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
            - --validating-webhook=:8443
            - --validating-webhook-certificate=/usr/local/certificates/cert
            - --validating-webhook-key=/usr/local/certificates/key

Delete the Service from manifest

You will need to entirely delete the Service of type LoadBalancer named: ingress-nginx from the manifest as you will already be using hostPort.

After this steps you should be able to use Ingress resources and communicate with them on VM_IP:80/443.

Additional resources:

user_122571340
Related