I am pretty confused on the best way to do what I am trying to do.
I have a series of jobs that take a lot of time to process and so I would like to initialize a substantial pool of "workers" on app start up that will be able to handle a request when free.
Breakdown of the process:
- Create 10 classifiers (workers) on app start up and keep them idle somewhere.
- When app started a request comes through Flask POST command.
- Request is passed to an available classifier.
- Classifier returns result of job.
How would I go about doing this?
EDIT: It is worth noting that the classifiers take a substantial amount of time to start up and so will need to be available and already running when work it passed to them.
class View(views.MethodView):
def get(self):
return render_template('index.html')
def post(self):
app_form_elements = request.form
#Assumption button clicked on browser interface...
jobs = ["job one", "job two", "job three"]
for job in jobs:
#Send each job to next available classifier pool.
return self.get()
app.add_url_rule('/', view_func=View.as_view('main'), methods=['GET','POST'])
app.debug = True
if __name__ == '__main__':
app.run(threaded=True)
EDIT:
The classifier setup is something like this:
class Classifier():
"""
Class will take in a classifier and a test data set and print out the overall accuracy.
"""
def __init__(self):
self.load = self.toSomeStuff()
print('classifier initialised.\n')
def doSomeWork(self):
#Initialised classifier objects called with work to do.
initialise_classifier = Classifier()
#the jobs
initialise_classifier.doSomeWork()
So basically I need a pool of pre-initialized classifiers and then be able to call the "doSomeWork" function on them with each job that comes in via the post method.