I am testing two Google Cloud products, namely: Cloud Functions and Cloud Run. I am leaning toward using cloud run given that it allows you for more customization and observability.
However one thing I noticed that for some reason, Cloud Functions are faster than Cloud Run, despite using the same code and configuration (2 GB RAM). Even increasing RAM or core numbers for Cloud Run didn't improve speed.
My main issue is model loading time (to get a fast cold start), in Cloud Function it takes 1/5 what it takes in Cloud Run, that's very detrimental for my use case.
I ran benchmark tests and this is a re-occurring pattern for many different models, does anyone have an idea on why this is the case ?
UPDATE: My dockerfile:
FROM python:3.8
# set a directory for the app
WORKDIR /usr/src/app
# copy all the files to the container
COPY . .
# install dependencies
RUN pip install --no-cache-dir -r requirements.txt
RUN pip install Flask gunicorn
RUN apt-get install util-linux
# run the command
CMD exec gunicorn --bind :$PORT --timeout 0 main:app