Remote ray call Ignoring function arguments

user_5869242 0
Viewed 35

I am trying to apply ray to a transformer pipeline as:

@ray.remote
def predict(pipeline, text_data, max_length, min_length, do_sample):
    return pipeline(text_data, max_length, min_length, do_sample)

and initializing as:

predictions = ray.get(predict.remote(pipe_1, section, max_length=some_length, min_length=some_length, do_sample=False))

the pipeline pipe_1 is defined using ray.put()

pipe_1 = ray.put("transformer pipeline method definition")

Pipeline method definition is similar to as described here. I have initialized ray.init() as : ray.init(num_cpus=num_cpus, ignore_reinit_error=True)

However its seems my *args are being ignored. Looks like max_length, min_length and do_sample arguments are ignored - below is the sample console output:

(predict pid=19440) Ignoring args : (529, 423, False)
(predict pid=19440) Ignoring args : (680, 544, False)

Any suggestions.

Source code and article: Parallel Inference of HuggingFace Transformers on CPUs

ray huggingface-transformers python-3.x python
1 Answers

Copy-pasting from discuss:

Sorry I’m not exactly sure what you mean by the args getting ignored. Could you post a runnable reproduction, preferably without the actual pipeline (you can just use an empty function)?

I’m not sure if this is related to your issue, but I would also suggest calling pipeline as a remote function instead of passing it into the wrapper task. This is likely to be more efficient and fault-tolerant, as Ray will broadcast the pipeline definition to all workers beforehand. You can do this like so:

pipe1 = "transformer pipeline method definition"
pipe1_remote_fn = ray.remote(pipe1)
pipe1_remote_fn.remote(section, max_length=some_length, min_length=some_length, do_s
user_110257170
Related