We have a problem with frequent restarts of App Engine instances which last for 15-30 minutes, sometimes maybe 1 hour.
Last 24 hours, we had 72 instance restarts. We have looked into the logs, but can't find any error messages explaining this.
Min_instances is set to 1.
The app is a PHP Codeigniter app running on the php73 runtime.
Maybe this is relevant as it shows up regularly in the log, not at the same time as web requests :
A 2020-05-01T17:46:46.675532Z [start] 2020/05/01 17:46:46.674713 Quitting on terminated signal
A 2020-05-01T17:46:46.900441Z [start] 2020/05/01 17:46:46.899377 Start program failed: termination triggered by nginx exit
Looking at the request log it looks like there is no pattern in page requests that could lead to instances crashing.
All page-requests load in typically 1-80 ms, there are no heavy scripts. It looks like the instances crash while idle.
We have also tried to increase the instance type to F4 with the same results.
The graphs for CPU usage and memory usage don't give us any clue.
The problem with this is loading requests for site visitors. Most of the time the site is fast and responsive, but it is possible with accidental loading times of 1s+ when new instances start. We have set up warm up requests, but that does not cover all instance starts.
Is this a normal behavior ? How can we debug further ? Any clue what can be wrong ?
Thanks for any help.
EDIT: Here is our app.yaml:
runtime: php73
entrypoint: serve public_html/index.php
instance_class: F2
automatic_scaling:
min_instances: 1
inbound_services:
- warmup
vpc_access_connector:
name: "xx"
handlers
- url: /
script: auto
secure: always
- url: /(.+)
script: auto
secure: always
env_variables:
CLOUD_SQL_CONNECTION_NAME: xx
REDIS_HOST: xx
REDIS_PORT: xx