Slow Database Access from GCP Cloud Run

Calling the POST method in the following code takes about 35ms when run on a GCP virtual machine (as a container), but when deployed to GCP Cloud Run (as a container) it runs much slower, taking about 6500ms, almost 200x slower.

I generously increased the Cloud Run CPU and memory allocations but the speed remained the same. I narrowed it down to the database code, by putting it in a loop of 10 times, which made it almost 10 times slower.

In both cases there are 10 connections to the database. However, in the Cloud Run case, the connections build up slowly, starting with 1 then 2 more after a few seconds, and then it waits for requests and adds another 2 every request until 10. In contrast, when running it on the VM (as a container) it instantly establishes 10 connections and nothing changes with the requests.

Any idea why such a drastic difference in performance? I'm sure it's something I'm doing wrong as Cloud Run would be almost useless if this is normal.

The database is a Cloud SQL postgresql instance.

Update 1: I should point out that I am not using the instance unix socket. Could that be relevant?

Update 2: Adding this extra line of code does not fix the problem:

addDataSourceProperty("unixSocketPath", "/cloudsql/valiant-index-344017:asia-southeast1:main")

Update 3: I'm building this using Jib and uploading the image to gcr.io then deploying to cloud run. See build.gradle.kts after the source code.

Update 4: For completeness I'm also appending src/main/resources/application.conf (after the first two files below) which is needed to make the code run.

Following is the full program App.kt:

package com.example

import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import io.ktor.server.application.*
import io.ktor.server.plugins.contentnegotiation.*
import io.ktor.server.request.*
import io.ktor.server.response.*
import io.ktor.server.routing.*
import io.ktor.serialization.kotlinx.json.*
import kotlinx.serialization.*
import kotlinx.serialization.json.*
import org.jetbrains.exposed.sql.Database
import org.jetbrains.exposed.sql.Table
import org.jetbrains.exposed.sql.insert
import org.jetbrains.exposed.sql.transactions.transaction
import org.postgresql.util.PSQLException


@Serializable
data class Message (val id: Int, val message: String)

object Foo: Table() {
  val n = integer("n")
}

fun Application.main() {
  HikariConfig().run {
    jdbcUrl = "jdbc:postgresql:///main"
    username = "main"
    password = "yourpasswordhere"
    addDataSourceProperty("socketFactory", "com.google.cloud.sql.postgres.SocketFactory")
    addDataSourceProperty("cloudSqlInstance", "valiant-index-344017:asia-southeast1:main")
    HikariDataSource(this)
  }.let {
    Database.connect(it)
  }
  install(ContentNegotiation) {
    json(Json {
      prettyPrint = true
      isLenient = true
    })
  }
  routing {
    get("/") {
      call.respondText("hello")
    }
    post("/") {
      try {
        val message = call.receive<Message>()
        try {
          (0 until 10).forEach { i ->   // ***** THIS CODE BLOCK IS SLOW *****
            transaction {
              Foo.insert {
                it[n] = message.id * 10 + i
              }
            }
          }
          call.respond<Message>(Message(message.id, "ok"))
        } catch (e: PSQLException) {
          println("exception: $e")
          call.respond<Message>(Message(0, "exception: $e"))
        }
      } catch (e: Throwable) {
        println("err: $e")
        call.respondText("err: $e")
      }
    }
  }
}

Update 3: build.gradle.kts is below. I'm using Jib not a Dockerfile.

plugins {
    kotlin("jvm") version "1.7.10"
    kotlin("plugin.serialization") version "1.7.10"
    id("com.google.cloud.tools.jib") version "2.1.0"
    application
}

repositories {
    mavenCentral()
}

dependencies {
    implementation("io.ktor:ktor-server-core:latest.release")
    implementation("io.ktor:ktor-server-netty:latest.release")
    implementation("io.ktor:ktor-server-content-negotiation:latest.release")
    implementation("io.ktor:ktor-serialization-kotlinx-json:latest.release")
    implementation("com.google.cloud.sql:postgres-socket-factory:latest.release")
    implementation("com.zaxxer:HikariCP:latest.release")
    implementation("org.postgresql:postgresql:latest.release")
    implementation("org.jetbrains.exposed:exposed-core:latest.release")
    implementation("org.jetbrains.exposed:exposed-dao:latest.release")
    implementation("org.jetbrains.exposed:exposed-jdbc:latest.release")
    implementation("ch.qos.logback:logback-classic:latest.release")
}

application {
    mainClass.set("io.ktor.server.netty.EngineMain")
}

kotlin.sourceSets.all {
    languageSettings.optIn("kotlin.RequiresOptIn")
}

jib {
    to.image = "gcr.io/valiant-index-344017/bgcl/wordchase_host"
    container.mainClass = "io.ktor.server.netty.EngineMain"
}

Update 4: src/main/resources/application.conf below for completeness:

ktor {
  deployment {
    port = ${PORT}
  }
  application {
    modules = [ com.example.AppKt.main ]
  }
}