How can I get model inferences in parallel using ray without loading the model each time?

Viewed 226

I'm new to ray and was experimenting with it. I have a predictor class with a tensorflow model and want to use the predict method of the class to obtain inferences in parallel in another function.

class Predictor:
    def__init__(...):

    def load_models(...):

    def load_data(...):
    
    def predict(...):

In another script I have initialized the class and loaded the data and models needed. Then I tried to call a function

@ray.remote
def newfunc(predictor,inputs):
   # creating dataset
   # preprocessing
   outputs = predictor.predict(inputs)

   return outputs

if __name__ == __main__:
    predictor = Predictor()
    predictor.load_models()
    predictor.load_data()



    print("testing ray")
    refs = [newfunc.remote(predictor,x) for x in np.array_split(bigdata,24)]
    print(ray.get(refs[0]))

I'm getting an error saying my object doesn't have a certain attribute AttributeError: 'Predictor' object has no attribute 'dataManager'

However I can use the instance of the class just fine without ray and this issue won't come up. Does Ray create a new object with each worker? It doesn't seem to be loading the datamanager each time so maybe that's why I get the attribute error. I was hoping to only have to create the object once and have everything loaded once and use that as an argument. I've tried creating an actor of the object but no luck with that approach. Does anyone have any suggestions on how to parallelize a function that uses an object?

I've also tried this:

@ray.remote
def newfunc(predictor,inputs):
   # creating dataset
   # preprocessing
   outputs = predictor.predict(inputs)

   return outputs

if __name__ == __main__:
    predictor = Predictor()
    predictor.load_models()
    predictor.load_data()
    predictor_id = ray.put(predictor)


    print("testing ray")
    refs = [newfunc.remote(predictor_id,x) for x in np.array_split(bigdata,24)]
    print(ray.get(refs[0]))

But that hasn't worked either. In fact I tried accessing the attributes from the original object and using the ray objectref

>>> predictor.dataManager
<prophetball.StreamLearner.utils.generators.DataManager object at 0x7fac0c00c290>
>>> ray.get(predictor_id).dataManager
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Predictor' object has no attribute 'dataManager'

Full traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/anaconda3/envs/LarusTF/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 47, in wrapper
    return func(*args, **kwargs)
  File "/root/anaconda3/envs/LarusTF/lib/python3.7/site-packages/ray/worker.py", line 1431, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::promos_in_chromosome() (pid=7750, ip=172.16.69.99)
  File "python/ray/_raylet.pyx", line 502, in ray._raylet.execute_task
  File "<stdin>", line 133, in promos_in_chromosome
  File "/run/media/root/prophet/DevOps/bnlwe-da-p-80200-prophetball/prophetball/StreamLearner/utils/predictor.py", line 1336, in units_from_predictor
  File "/run/media/root/prophet/DevOps/bnlwe-da-p-80200-prophetball/prophetball/StreamLearner/utils/predictor.py", line 1194, in add_missing_features_to_forecast
    df = self.dataManager.add_non_POS_data(df,lock_date = lock_date)
AttributeError: 'Predictor' object has no attribute 'dataManager'
0 Answers
Related