Cannot install kube-prometheus-stack in k8s 1.15

user_363603 0
Viewed 3981

Running kubernetes 1.15 in azure.

I need a basic alert (e-mail/slack notification) when one or more of my applications/pods are down in kubernetes.

As an example I have https://cert-manager.io/docs/ running in multiple clusters (hosted in azure) and I would like to get an alert (e-mail/slack notification) if it stops running.

Based on this post:

How do I set up a hook to send an email on Kubernetes pod restart?

it seems to get an e-mail alert I need to install Prometheus + Grafana access the web-ui and configure alerts so based on:

https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack

https://artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack

I have tried:

helm version
version.BuildInfo{Version:"v3.1.2", GitCommit:"d878d4d45863e42fd5cff6743294a11d28a9abce", GitTreeState:"clean", GoVersion:"go1.13.8"}


helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm repo update

helm install kube-prometheus-stack prometheus-community/kube-prometheus-stack --namespace monitoring

But that gives:

Error: failed to install CRD crds/crd-alertmanager.yaml: unable to recognize "": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1"

Here there is some guide on how to create the crds manually:

https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#helm-fails-to-create-crds

but that should only be necessary if running helm 2.x which I am not I am running 3.1.2.

Also if I try to install them manually I get:

$ kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/release-0.42/example/prometheus-operator-crd/monitoring.coreos.com_alertmanagers.yaml
error: unable to recognize "https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/release-0.42/example/prometheus-operator-crd/monitoring.coreos.com_alertmanagers.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1"
$ kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/release-0.42/example/prometheus-operator-crd/monitoring.coreos.com_podmonitors.yaml
error: unable to recognize "https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/release-0.42/example/prometheus-operator-crd/monitoring.coreos.com_podmonitors.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1"
$ kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/release-0.42/example/prometheus-operator-crd/monitoring.coreos.com_prometheuses.yaml
error: unable to recognize "https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/release-0.42/example/prometheus-operator-crd/monitoring.coreos.com_prometheuses.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1"
...

Also I found this kube-prometheus stack compatibility matrix:

https://github.com/prometheus-operator/kube-prometheus#compatibility

but versions in that matris does not match the ones I get:

$ helm search repo prometheus-community/kube-prometheus-stack --versions
NAME                                        CHART VERSION   APP VERSION DESCRIPTION                                       
prometheus-community/kube-prometheus-stack  10.1.2          0.42.1      kube-prometheus-stack collects Kubernetes manif...
prometheus-community/kube-prometheus-stack  10.1.1          0.42.1      kube-prometheus-stack collects Kubernetes manif...
prometheus-community/kube-prometheus-stack  10.1.0          0.42.1      kube-prometheus-stack collects Kubernetes manif...
prometheus-community/kube-prometheus-stack  10.0.2          0.42.1      kube-prometheus-stack collects Kubernetes manif...
prometheus-community/kube-prometheus-stack  10.0.1          0.42.1      kube-prometheus-stack collects Kubernetes manif...

So seems that might be a 3rd way to install Prometheus

Any input appreciated.

UPDATE:

Randomly selecting the previous major version (9.4.10) seems to work:

$ helm install kube-prometheus-stack prometheus-community/kube-prometheus-stack --namespace monitoring --version 9.4.10
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
manifest_sorter.go:192: info: skipping unknown hook: "crd-install"
NAME: kube-prometheus-stack
LAST DEPLOYED: Fri Oct 23 15:15:03 2020
NAMESPACE: monitoring
STATUS: deployed
REVISION: 1
NOTES:
kube-prometheus-stack has been installed. Check its status by running:
  kubectl --namespace monitoring get pods -l "release=kube-prometheus-stack"

Visit https://github.com/prometheus-operator/kube-prometheus for instructions on how to create & configure Alertmanager and Prometheus instances using the Operator.

Guess trial and error is the way to go when installing stuff on older k8s version, could be great with compatibility matrices though.

kubernetes alert monitoring
1 Answers

Based on the kube-prometheus-stack repo, this helm chart is restricted for K8s versions 1.16.0 or above;

kubeVersion: ">=1.16.0-0"

Even though the github README says the prerequisites as Kubernetes 1.10+ with Beta APIs, internally the helm chart checks for the kube version to be 1.16.0 or above.

So I believe, you will need to try this on an upgrade K8s cluster.

If upgrading the cluster is not an option, maybe you could try the deprecated old version of this.

https://github.com/helm/charts/tree/master/stable/prometheus

user_25811280
Related