How can you simulate "serverless" Cloud SQL?

user_13668802 0
Viewed 846

Problem: Cloud SQL instances run indefinitely and are monetarily expensive to host.

Goal: Save money while not compromising on database availability.

It has been almost four years and Google Cloud has not fulfilled this feature request that has already been implemented on AWS with their Aurora RDS.

Since it does not seem that on demand Cloud SQL that auto-scales to zero is coming any time soon, will the following strategy work?

  1. Have instances of Cloud SQL, a Baby and a Papa. They follow the master/slave replica principle, with twist. The Baby instance is small with few vCPU's and low memory, it always runs, but does so cheaply. However, the Papa instance is expensive with high vCPU and high memory but runs only when needed.
  2. To begin, only the Baby Cloud SQL instance is running so it is the master that accepts reads/writes. The Papa Cloud SQL instance is not running.
  3. Since I am using standard app engine that will auto-scale to zero with no traffic, schedule a cron job that checks every 10 min if no app engine instances exists. In this case, the application has no traffic. If this is not the case, the Papa Cloud SQL instance is started. Once started, the Papa instance becomes the master that accepts reads/writes while the Baby instance becomes a slave replica capable of only reads.
  4. If the cron job detects the app engine has zero instances running, this means there is no traffic. Thus, the Papa Cloud SQL instance is stopped and the Baby Cloud SQL replica is promoted to master and can accept reads/writes.
  5. In this way, the expensive Papa instance runs on demand. If there is a traffic spike when the Papa instance is stopped or rebooting, the Baby instance will still be able to respond to requests.

This strategy ensures that the expensive Papa Cloud SQL instance only runs with traffic. Is this Baby-Papa dynamic possible on Google Cloud?

google-cloud-sql google-cloud-functions google-cloud-storage google-cloud-platform google-app-engine
1 Answers

Cloud SQL has an Admin API that can be used to manipulate your Cloud SQL instances in such a way. You could build pieces of what you are describing using Cloud Scheduler to trigger a Cloud Function which uses the API to start and stop instances, or even promote/demote them to master.

However, it's probably a bad idea. These operations can take several minutes to complete and would give you dramatic increases to cold start times for requests. Additionally, SQL servers prefer to be long running for a reason - they use resources to cache and optimize queries to improve performance. Start, stoping, and resizing instances can cause you to lose these benefits.

It's better to consider - do you actually need a relational database? If not, it's probably better to use something like Firestore, which is a serverless product.

If you determine that you do indeed need a relational database, can you optimize your use for a smaller Cloud SQL instance? Can you cache queries using Memorystore or Firestore as listed above, or instead use the services I described above to export the results on a timed basis, which would be easier for your app to consume?

Would it be better to start and stop your Cloud SQL instance when there is no traffic? If you traffic is based around certain predictable times, you could schedule your instance to resize at the start and stop of these time periods.

Finally, if cost is really an option, you could run your own SQL server on a GCE instance. This means you have to do pretty much all of the management yourself (install, updates, maintenance, etc), but it would be cheaper.

All of these are probably much more functional solutions than trying to shoehorn non-serverless infrastructure to match a serverless workload.

user_94148030
Related