TypeError: from_pretrained() got an unexpected keyword argument 'file_name'

user_19762134 0
Viewed 78

I'm trying to quantize a seq2seq model (M2M100) using optimum library provided by Huggingface. As per this guide, I'm trying to quantize the encoder and decoder one by one but that requires me to overwrite the model name. Following the documentation in the guide, I used the code:

encoder_quantizer = ORTQuantizer.from_pretrained(model_dir, file_name="encoder_model.onnx")

This code is throwing the following error:

TypeError: from_pretrained() got an unexpected keyword argument 'file_name'

I tried examining ORTQuantizer.from_pretrained and got the following:

<function optimum.onnxruntime.quantization.ORTQuantizer.from_pretrained(model_name_or_path: Union[str, os.PathLike], feature: str, opset: Optional[int] = None) -> 'ORTQuantizer'>

Clearly, from_pretrained here doesn't have a file_name parameter as has been indicated in the guide. Can someone please help me debug this error? Thanks!

seq2seq huggingface-transformers
1 Answers

Posting the solution I found while exploring optimum GitHub repo. The problem is that installing optimum via pip is downloading v1.3 which did not have the fix for quantizing seq2seq models. Instead install the package directly from GitHub using the command below. It worked fine afterwards.

python -m pip install git+https://github.com/huggingface/optimum.git
user_197621340
Related