AKS with Static IP and Custom Cert / AKS Ingress issues

Admin 1
Viewed 1500

Well, for the last 2 days I battled this documentation:

https://docs.microsoft.com/en-au/azure/aks/static-ip and https://docs.microsoft.com/en-au/azure/aks/ingress-own-tls

First of all I ensured that I had my aks k8s cluster upgraded to 1.11.5, so there is no question about having the static IP in a different resource group.

Overall, I could not get the static IP really working. With dynamic everything sounds fine, but I cannot add a A record for a dynamic IP.

I managed to deploy everything successfully, but any curl ip.. does not work. I did run exec -ti locally, and locally everything is fine.

Could someone please point me to a GitHub config or article that has this configuration running? As a disclaimer I know azure very well, so well the service principal assignments are well done, etc. However, I am new, only a few months on k8s.

Thanks in advance for any suggestion.

I can share logs if needed but believe I did check everything from dns to ingress routes. I am worried that this doc is not good and I am just loosing my time.

azure-aks kubernetes-ingress kubernetes azure
2 Answers

Answering myself this question, after quite a journey, for when I get older and I forget what I've done, and maybe my nephew will save some hours someday.

  1. First, it's important:

In the values provided to nginx-ingress chart template, there are 2 annotations that are important:

service.beta.kubernetes.io/azure-load-balancer-resource-group: "your IP's resource group" externalTrafficPolicy: "Local"

Here are all the values documented: https://github.com/helm/charts/blob/master/stable/nginx-ingress/values.yaml

The chart can be deployed near your service's namespace, it should not be in kube-system (with my current knowledge I don't find a reason to have it in system).

  1. Second, could be misleading

There is a delay of ~30+ seconds (in my case) from the moment when IP appeared in the kubectl get services --watch and till the moment curl -i IP was able to answer the call. So, if you have automation or health probes then ensure that you have 1 - 2 mins added to wait. Or maybe take better nodes, bare metal machines.

  1. Look at GCE and DO for the same setup as might help: https://cloud.google.com/community/tutorials/nginx-ingress-gke https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-with-cert-manager-on-digitalocean-kubernetes The guys at DO, are good writes as well.

Good luck!

Admin1

Based on your comments, it seems that you are trying to override the externalIPs but use the default value of the helm chart for controller.service.type which is LoadBalancer. What you might want to do is to keep controller.service.type to LoadBalancer and set controller.service.loadBalancerIP with your static IP instead of overriding externalIPs.

Here some documentation from microsoft.

user_7327710
Related