How to use VPA with database pod in kubernetes?

user_1885666 0
Viewed 667

I want to apply VPA vertical pod auto scaling for database pods. Can we use VPA for database auto scaling (Vertical) as VPA requires at least 2 replicas (ref : https://github.com/kubernetes/autoscaler/issues/1665#issuecomment-464679271) as it delete pods when set criteria is reached. So pods are deleted hence also data.

What is good practice for using VPA with database pods?

autoscaling google-kubernetes-engine kubernetes
2 Answers

As I understand it, the real question is how to run a stateful workload with multiple replicas.

Use StatefulSets to configure n replicas for a database. StatefulSet pods have stable names which are preserved across pod restarts (and reincarnations). Combined with PersistentVolumeClaim templates (accepted with StatefulSet spec) and headless services, it is capable of retaining same volumes and network FQDN across reincarnations.

Take a look at Helm charts for various databases, e.g. MySQL chart, for useful insights.

On a side note, it might be worthwhile to consider using an operator for the database application you're using. Operators for most applications can be found on https://operatorhub.io.

user_24106410

VPA - Vertical pod autoscaler can work in 2 ways:

  • Recommendation mode - it will recommend the requests and limits for pods based on resources used
  • Auto mode - it will automatically analyze the usage and set the request and limits on pods. This will result in pod termination to recreate it with new specification as stated here:

Due to Kubernetes limitations, the only way to modify the resource requests of a running Pod is to recreate the Pod. If you create a VerticalPodAutoscaler with an updateMode of "Auto", the VerticalPodAutoscaler evicts a Pod if it needs to change the Pod's resource requests.

Cloud.google.com: Kubernetes Engine: Docs: Concepts: Vertical pod autoscaler

Please refer to above link for more information regarding the concepts of VPA.

The fact that it needs at least 2 replicas is most probably connected with the fact of high availability. As the pods are getting evicted to support new limits they are unable to process the request. If it came to situation where there is only 1 replica at the time, this replica wouldn't be able to respond to requests when in terminating/recreating state.

There is an official guide to run VPA on GKE:

VPA supports: Deployments as well as StatefulSets.

StatefulSet

Like a Deployment, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling.

If you want to use storage volumes to provide persistence for your workload, you can use a StatefulSet as part of the solution.

Kubernetes.io: StatefulSet

Configuring StatefulSet with PersistentVolumes will ensure that the data stored on PV will not be deleted in case of pod termination.

To be able to use your database with replicas > 1 you will need to have replication implemented within your database environment.

There are guides/resources/solutions on running databases within Kubernetes environment. Please choose the solution most appropriate to your use case. Some of them are:

After deploying your database you will be able to run below command to extract the name of the StatefulSet:

  • $ kubectl get sts

You can then apply the name of the StatefulSet to the VPA like below:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: DB-VPA 
spec:
  targetRef:
    apiVersion: "apps/v1"
    kind:       StatefulSet
    name:       <INSERT_DB_STS_HERE>
  updatePolicy:
    updateMode: "Auto"

I encourage you also to read this article:

user_122571340
Related