Is Tensorflow_Hub Universal Sentence Encoder threadsafe?

user_3792705 0
Viewed 416

I want to use Django server to implement natural language search with Universal Sentence Encoder (USE) and Annoy based database. The loading of USE is very slow:

 import tensorflow as tf
 import tensorflow_hub as hub
 module_url = "https://tfhub.dev/google/universal-sentence-encoder/4"
 model = hub.load(module_url)

It's impossible to load pre-trained USE embedding on every browser request.

The plan is to create USE instance as a global object and store in session. Each request uses the object in session to access USE object to get embedding of query sentence.

Question:

Is Tensorflow_hub USE thread safe when getting embeddings for queries, since there is only 1 global USE object?

If running Django server in production Nginx, multiple workers will be configured. If Tensorflow is configured to use GPU, does mean only 1 worker can be configured for Nginx?

If it's not thread safe, what's good design pattern can be used here?

tensorflow-hub tensorflow thread-safety django
1 Answers

I think that to solve you problem you might want to look into TF Serving for your serving needs and it will take care of GPU, multiple workers and all the load issues. From your Django application you could just call TF Serving API.

user_4485470
Related