I've been successfully using the Google cloud_sql_proxy on multiple Compute Engine instances for some time, until today, one instance at a time, the proxy started to show the following error pattern:
2017/05/30 13:28:07 New connection for "project-id-1234:us-central1:sql_instance"
2017/05/30 13:28:07 couldn't connect to "project-id-1234:us-central1:sql_instance": Post https://www.googleapis.com/sql/v1beta4/projects/project-id-1234/instances/sql_instance/createEphemeral?alt=json: stream error: stream ID 1; PROTOCOL_ERROR
2017/05/30 13:28:41 New connection for "project-id-1234:us-central1:sql_instance"
2017/05/30 13:28:41 Thottling refreshCfg(project-id-1234:us-central1:sql_instance): it was only called 33.490705951s ago
2017/05/30 13:28:41 couldn't connect to "project-id-1234:us-central1:sql_instance": Post https://www.googleapis.com/sql/v1beta4/projects/project-id-1234/instances/sql_instance/createEphemeral?alt=json: stream error: stream ID 1; PROTOCOL_ERROR
When trying to connect directly to MySQL (while using the proxy) I get error 2013 (HY000):
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0 "Internal error/check (Not system error)"
What I've tried
- Restarting the cloud_sql_proxy yielded a temporary fix until finally both my Compute Engine instances are unable to connect to my Cloud SQL instance and the proxies show only this result.
- Restarting the Cloud SQL instance and both Compute Engine instances.
- Eliminating the proxy: I added the appropriate networks to my SQL instance's Authorized Networks, and updated all applications to use the public IP. This restored functionality to my production apps, but now I'm using a public connection instead of local/proxy.
Some research
- I came across a similar issue relating to Google Cloud SQL that yielded the same MySQL error above, but it appears to have only affected connecting to Cloud SQL from external, non GCE/GKE networks.
- A few others have reported the same issue also started for them this morning on the Google Cloud SQL Discuss group.