I am using Google-App-Engine standard (Not flex) Enviroment with Python2.7, and I need to load some pre-trained models (Gensim's Word2vec and Keras's LSTM).
I need to load it once (since it very slow - takes around 1.5 seconds) and keep it in faster access for several hours.
What is the best & fastest way to do so?
Thanks!
IMHO the best place for read-only data (including imported code!) needed to be accessed at any time by individual requests is the global application variables area.
Such variables would typically be loaded exactly once per GAE instance lifetime and available until the instance goes away.
Since loading of the data is expensive you need to be aware that it could impact the response time for requests coming in while the instance is starting up (i.e. while the loading request is still active). There are 2 ways to address this:
one would be to use "lazy" loading of the data - effective if just a small percentage of the incoming requests actually need the data. But the requests which actually need the data when it's not available will still be affected, so it'll just reduce the impact of the problem. The method is described in detail in the App Engine Startup time and the Global Variable problem article:
from google.appengine.ext import ndb
# a global variable
gCDNServer = None
def getCDN():
global gCDNServer
if gCDNServer==None:
gCDNServer = Settings.query(Settings.name == "gCDNServer").value
return gCDNServer
another approach, which would completely eliminate the problem, is to make your app support warmup requests (available only if you're using automatic scaling). The data would be loaded by the warmup request handler and will always be available for "live" requests (because no "live" requests will be routed to the instance until the warmup request handling completes).
It might be possible to add logic to drop the data from memory (to reduce the app's memory footprint) if/when you know it'll no longer be needed (i.e. after those several hours you mentioned expired), but that complicates the picture, especially if you configured your app as threadsafe. I'd simply separate the code which doesn't need the data from the one which does in different services and leave autoscaling shut down the instances with the global data when no longer needed.