How do I avoid unschedulable pods when running an autoscale-enabled Google Container/Kubernetes Engine cluster?

user_1858091 0
Viewed 13270

I'm running into an issue where I seem to max out my Container Engine (Kubernetes Engine) cluster at 9 nodes (or 9 vCPUs). For context, I'm trying to run around 50 or so web scrapers as Kubernetes Cron Jobs. Here is the gcloud command I am using to create the cluster in the us-east1-c zone:

gcloud beta container clusters create my-example-cluster \
  --cluster-version=1.8.1-gke.1 \
  --machine-type=f1-micro \
  --enable-autoscaling \
  --min-nodes=1 \
  --max-nodes=60

When I checked out IAM & Admin > Quotas, it looked like I was possibly maxing out on in-use IP addresses (max=8).

After launching all the scrapers, I'm ending up around 35 pods that are unschedulable (seemingly because I can't exceed 9 nodes?).

Has anyone run into a similar situation and come up with a solution? Is there a way, perhaps via Kubernetes config, to not count against IP address usage since these nodes do not need to be accessible externally (if that's even the issue)? Also open to other setups if anyone has recommendations.

Thanks for all the help!

P.S. I do have billing enabled

google-kubernetes-engine kubernetes google-cloud-platform
1 Answers

I received some help on this issue from the Google Cloud Platform Community Slack. It turns out I was hitting the In-use IP addresses quota; however, I didn't realize you could request increased quotas.

I was able to increase quotas by:

  1. Navigating to Menu > IAM & admin > Quotas in the Google Cloud Platform console for my project
  2. Selecting/checking the services in the region I was using (e.g., In-use IP addresses in us-west1)
  3. Click EDIT QUOTAS button above the table of services
  4. Input/verify contact information
  5. Comply with GCP support when they reach out to you

So, basically an oversight on my end but maybe it will help somebody else who didn't notice or wasn't aware of that option.

user_18580910
Related