So, I have a flask application that loads some data from S3 into memory. Subsequent requests use this data and there's no updates to this data except for a daily cron job which refreshes it.
Now, what I've noticed is that, this background process (which loads the data into memory at startup) is usually executed by uWSGI's master process (5 mins after application startup - i can't do it any sooner because pod deployment is somehow affected if the background process is kicked off immediately). But, workers & threads have been spawned by uWSGI when the application started up so they don't have a copy of this data. Hence, any requests served by these workers & threads return a 404 because they don't have this data!!
How can I "sync" the data amongst all these threads & processes?
Here's what my app.ini looks like:
# The following article was referenced while creating this configuration
# https://www.techatbloomberg.com/blog/configuring-uwsgi-production-deployment/
# Please make changes according to your application's expected load, etc
[uwsgi]
strict = true ; Only valid uWSGI options are tolerated
master = true ; The master uWSGI process is necessary to gracefully re-spawn and pre-fork workers,
; consolidate logs, and manage many other features
enable-threads = true ; To run uWSGI in multithreading mode
vacuum = true ; Delete sockets during shutdown
single-interpreter = true ; Sets only one service per worker process
die-on-term = true ; Shutdown when receiving SIGTERM (default is respawn)
need-app = true ; Prevents uWSGI from starting if it is unable to find or load your application module
;disable-logging = true ; By default, uWSGI has rather verbose logging. Ensure that your
;log-4xx = true ; application emits concise and meaningful logs. Uncomment these lines
;log-5xx = true ; if you want to disable logging
# Logging for UWSGI
req-logger = file:/opt/docker/logs/application.log
logger = file:/opt/docker/logs/application.log
logformat = %(ltime) : PID %(pid) : %(proto) : %(uagent) : %(method) : %(uri) : %(status)
; (ltime) : Human readable timestamp (Apache format)
; (pid) : Worker pid
; (proto) : Protocol
; (uagent): User Agent
; (method): Request method
; (uri) : Request URI
cheaper-algo = busyness
processes = 2 ; Maximum number of workers allowed
threads = 10 ; Threads per process
cheaper = 1 ; Minimum number of workers allowed - default 1
cheaper-initial = 1 ; Workers created at startup
cheaper-overload = 1200 ; Will check busyness every 1200 seconds
cheaper-busyness-max = 2400 ; maximum busyness we allow
cheaper-step = 1 ; How many workers to spawn at a time
And this is what the scheduling looks like:
@inject
def schedule(
app,
my_service: MyService = Provide[
Container.my_service
],
):
scheduler = BackgroundScheduler(timezone=app.container.config.get("app.timezone"))
# schedule data fetch and run now
scheduler.add_job(
my_service.fetch_data,
CronTrigger.from_crontab(
app.container.config.get("app.my_service.data_reload_cron_utc"),
"UTC",
),
next_run_time=datetime.now() + timedelta(minutes=5),
)
logging.info(f"schedule(): my_service.fetch_data {job_name} added to scheduler")
scheduler.start()
logging.info("schedule(): scheduler started")
app.scheduler = scheduler