Load trained model on another machine - fastai, torch, huggingface

user_10947997 0
Viewed 495

I am using fastai with pytorch to fine tune XLMRoberta from huggingface. I've trained the model and everything is fine on the machine where I trained it.

But when I try to load the model on another machine I get OSError - Not Found - No such file or directory pointing to .cache/torch/transformers/. The issue is the path of a vocab_file.

I've used fastai's Learner.export to export the model in .pkl file, but I don't believe that issue is related to fastai since I found the same issue appearing in flairNLP.

It appears that the path to the cache folder, where the vocab_file is stored during the training, is embedded in the .pkl file: enter image description here

The error comes from transformer's XLMRobertaTokenizer __setstate__:

def __setstate__(self, d):
    self.__dict__ = d
    self.sp_model = spm.SentencePieceProcessor()
    self.sp_model.Load(self.vocab_file)

which tries to load the vocab_file using the path from the file.

I've tried patching this method using:

pretrained_model_name = "xlm-roberta-base"
vocab_file = XLMRobertaTokenizer.from_pretrained(pretrained_model_name).vocab_file

def _setstate(self, d):
    self.__dict__ = d
    self.sp_model = spm.SentencePieceProcessor()
    self.sp_model.Load(vocab_file)

XLMRobertaTokenizer.__setstate__ = MethodType(_setstate, XLMRobertaTokenizer(vocab_file))

And that successfully loaded the model but caused other problems like missing model attributes and other unwanted issues.

Can someone please explain why is the path embedded inside the file, is there a way to configure it without reexporting the model or if it has to be reexported how to configure it dynamically using fastai, torch and huggingface.

fast-ai huggingface-transformers pytorch nlp
1 Answers

I faced the same error. I had fine tuned XLMRoberta on downstream classification task with fastai version = 1.0.61. I'm loading the model inside docker.

I'm not sure about why the path is embedded, but I found a workaround. Posting for future readers who might be looking for workaround as retraining is usually not possible.

  1. I created /home/.cache/torch/transformer/ inside the docker image.
RUN mkdir -p /home/<username>/.cache/torch/transformers
  1. Copied the files (which were not found in docker) from my local /home/.cache/torch/transformer/ to docker image /home/.cache/torch/transformer/
 COPY filename:/home/<username>/.cache/torch/transformers/filename
user_90328070
Related