I have a substantial sqlite database file (~2 million rows) which I query in a Rust application using rusqlite on Google Cloud Run, using a bundled version of sqlite.
I do not add or write data, and open the database connection in read-only mode. Running the container in Docker locally, queries work perfectly. In Cloud Run, the database is successfully connected to, but attempting to prepare a query panics with "Error preparing search statement: SqliteFailure(Error { code: NotADatabase, extended_code: 26 }, Some("file is not a database"))".
This is a little bit mystifying because the database file has already been connected to! I had a few hypotheses:
- Something to do with a shortage of memory or attempting to write to Cloud Run's read-only file system? However, running in Docker with 256mb of memory and read-only mode enabled still worked.
- A syscall prevented by gvisor? Running strace in docker showed no problems though, and there are no unsupported syscall logs.
Any suggestions as to what might be happening?
EDIT: just to add to the mystery: I wrote a quick Flask application to do the same thing using Python's built in sqlite, and get the same error...
EDIT 2: Using a database with a single row instead of 2 million, everything works as expected.
The original database is around 435MB in size.
To have a minimally reproducible example, the Python version simply receives a request, opens a connection, and queries:
@app.route('/detail/<int:rowid>/', methods=["GET"])
def get_individual_address(rowid):
conn = sqlite3.connect(DB_NAME)
cur = conn.cursor()
cur.execute("""SELECT
postcode,
town_city_or_rural_location,
suburb,
line_one
FROM addresses WHERE id = ?""", (rowid,))
if result := cur.fetchone():
conn.close()
return {
"postcode": result[0],
"town_city_or_rural_location": result[1],
"suburb": result[2],
"line_one": result[3]
}
else:
conn.close()
abort(404)
Similarly a simplified Dockerfile which still reproduces the problem is below:
FROM python:3.9-slim-buster
COPY requirements.txt /
RUN pip install -r requirements.txt
COPY . /app
EXPOSE 8080
ENV PORT 8080
WORKDIR /app
CMD exec gunicorn --bind :$PORT address_lookup:app --workers 1 --threads 1 --timeout 60