Local Deployment/Installation of Kubeflow on Windows

user_8863398 0
Viewed 169

I am facing a major problem in installing Kubeflow locally on my Windows 10 Machine.

Machine Specs - OS: Windows 10, RAM: 16GB

Approaches Tried To Install

  1. Microk8s - Not Successful

    I get Cannot Install MicroK8s properly, due to “MicroK8s not found error”, along with instance crashing.

  2. MiniKube + Vagrant with VirtualBox - Partially Successful

    “Vagrant up” is very slow. And sometimes when I manage to open the Kubeflow console locally it crashes after running a couple of experiments. Errors arise from time-to-time and it is hard to pinpoint as to why they occur.

  3. Kind - Not Successful

    Out-dated docs and tutorials. Old commands don’t work and the manifests have been moved to another repo.

  4. K3s - Not Successful

    Cannot install manifests as it has been moved to another repo. And no updated docs directing as to how we can install it.

Resources Referred:

  1. https://kirenz.github.io/codelabs/codelabs/kubeflow-install/#4
  2. https://www.kubeflow.org/docs/components/pipelines/installation/localcluster-deployment/
  3. https://github.com/kubeflow/manifests
  4. And official doc of all the Approaches taken

Digging deep what I found was that some guides say that Kubeflow 1.5.0 is not compatible with version 1.22 and onwards. And as of now there are no older releases for Kubernetes(lower than 1.22) in the official site. Is this the root cause for the issues that I am facing?

Are there any other way to install and setup Kubeflow locally for Windows? It is hard to find a guide/tutorial or a video which is not outdated.

kubeflow kubernetes production windows
1 Answers

The current https://www.kubeflow.org/docs/started/installing-kubeflow/ page suggests using a package.

None of the packages are expressly for Windows. The "Charmed Kubeflow" looked promising for an install on a Windows machine, so I went with it. Here are the steps I figured out after much trial and error.

  1. Enable Hyper V for Windows. (If you have Windows Home, see https://www.makeuseof.com/install-hyper-v-windows-11-home/.)

  2. https://multipass.run/install. Choose Hyper V over Virtual Box if you can. If you cannot, then finish the install while ignoring any VirtualBox related error, and then do Command prompt, multipass set local.driver=hyperv. (You may need to restart your computer here.)

  3. Command prompt: multipass shell

  4. Shell: exit

  5. Command prompt: multipass stop

  6. In the Windows program "Hyper-V Manager", select the VM. Settings:

    • Memory, RAM: 4096, Enable Dynamic Memory.
    • Processor, Number of Virtual Processors = 2.
    • SCSI Controller, Hard Drive, Virtual hard disk, Edit, Action: Expand, New size 50 GB.

  7. Command prompt:

    multipass start
multipass shell

  8. Shell: (These steps are mostly from https://charmed-kubeflow.io/docs/quickstart.)

    sudo snap install microk8s --classic --channel=1.21/stable
sudo usermod -a -G microk8s $USER
newgrp microk8s
sudo chown -f -R $USER ~/.kube
microk8s enable dns
microk8s enable storage
microk8s enable ingress
microk8s enable metallb:10.64.140.43-10.64.140.49
microk8s enable dashboard

  9. Check the status until those items are enabled. Shell: microk8s status --wait-ready

  10. Shell:

    (Note: When I tried to enable istio before doing juju bootstrap microk8s, the juju bootstrap command consistently failed with the following error regardless of how much memory I allocated to the VM: failed to bootstrap model: creating controller stack: creating statefulset for controller: timed out waiting for controller pod: unschedulable: 0/1 nodes are available: 1 Insufficient memory.)

    sudo snap install juju --classic
juju bootstrap microk8s
microk8s enable istio
juju add-model kubeflow
juju deploy kubeflow-lite --trust
juju config dex-auth public-url=http://10.64.140.43.nip.io
juju config oidc-gatekeeper public-url=http://10.64.140.43.nip.io
juju config dex-auth static-username=admin
juju config dex-auth static-password=admin
watch -c juju status --color

  11. To access the Kubernetes dashboard: The Charmed Kubeflow Quickstart instructions for this will not work as-is for your Windows web browser. Try this:

    • Shell: microk8s dashboard-proxy. (This will keep running to serve the dashboard until you Ctrl-C cancel, close the window, or shutdown the VM.)
    • Command prompt: multipass list
    • Windows web browser: https://<ip address for the VM from multipass list>:<port number from dashboard-proxy>, copy-paste the token from dashboard-proxy

  12. To access the Kubeflow dashboard:

    • Shell: microk8s kubectl port-forward -n istio-system service/istio-ingressgateway 8080:80 --address=0.0.0.0. (This will keep running to serve the dashboard until you Ctrl-C cancel, close the window, or shutdown the VM.)
    • Command prompt: multipass list
    • Windows web browser: https://<ip address for the VM from multipass list>:8080
user_42707710
Related