In HuggingFace, every time I call a pipeline() object, I get a warning:
`"Setting `pad_token_id` to `eos_token_id`:{eos_token_id} for open-end generation."
How do I suppress this warning without suppressing all logging warnings? I want other warnings, but I don't want this one.
The warning comes for any text generation task done by HuggingFace. This is explained here, and you can see the code here. You can avoid that warning by manually setting the pad_token_id to the eos_token_id.
That is when you call
model.generate(**encoded_input)
just change it to
model.generate(**encoded_input, pad_token_id=tokenizer.eos_token_id)
and that will get rid of the error. However, I haven't found a way to set this directly from the pipeline interface. I'm guessing you could pass in some arguments to the ArgumentHandler. But I haven't tried it.