I am using Huggingface to further train a BERT model. I saved the model using two methods: step (1) Saving the entire model using this code: model.save_pretrained(save_location), and step (2) save the state_dict of the model using this code: torch.save(model.state_dict(),'model.pth')
However, when I try to load this pretrained BERT model using the following code bert_mask_lm = BertForMaskedLM.from_pretrained('save_location') for step (1) and torch.load('model.pth') for step (2), I am getting this following error in both steps:
AttributeError Traceback (most recent call last)
~/anaconda3/lib/python3.6/site-packages/torch/serialization.py in _check_seekable(f)
307 try:
--> 308 f.seek(f.tell())
309 return True
AttributeError: 'torch._C.PyTorchFileReader' object has no attribute 'seek'
During handling of the above exception, another exception occurred:
Detailed stacktrace of step (1) is as follows:
AttributeError Traceback (most recent call last)
~/anaconda3/lib/python3.6/site-packages/torch/serialization.py in _check_seekable(f)
307 try:
--> 308 f.seek(f.tell())
309 return True
AttributeError: 'torch._C.PyTorchFileReader' object has no attribute 'seek'
During handling of the above exception, another exception occurred:
AttributeError Traceback (most recent call last)
~/anaconda3/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1037 try:
-> 1038 state_dict = torch.load(resolved_archive_file, map_location="cpu")
1039 except Exception:
~/anaconda3/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
593 return torch.jit.load(opened_file)
--> 594 return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
595 return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
~/anaconda3/lib/python3.6/site-packages/moxing/framework/file/file_io_patch.py in _load(f, map_location, pickle_module, **pickle_load_args)
199
--> 200 _check_seekable(f)
201 f_should_read_directly = _should_read_directly(f)
~/anaconda3/lib/python3.6/site-packages/torch/serialization.py in _check_seekable(f)
310 except (io.UnsupportedOperation, AttributeError) as e:
--> 311 raise_err_msg(["seek", "tell"], e)
312 return False
~/anaconda3/lib/python3.6/site-packages/torch/serialization.py in raise_err_msg(patterns, e)
303 + " try to load from it instead.")
--> 304 raise type(e)(msg)
305 raise e
AttributeError: 'torch._C.PyTorchFileReader' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
~/work/algo-FineTuningBert3/FineTuningBert3.py in <module>()
1 #Model load checking
----> 2 loadded_model = BertForMaskedLM.from_pretrained('/cache/raw_model/')
~/anaconda3/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
1039 except Exception:
1040 raise OSError(
-> 1041 f"Unable to load weights from pytorch checkpoint file for '{pretrained_model_name_or_path}' "
1042 f"at '{resolved_archive_file}'"
1043 "If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. "
OSError: Unable to load weights from pytorch checkpoint file for '/cache/raw_model/' at '/cache/raw_model/pytorch_model.bin'If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
I am using the latest torch (1.7.1) and transformers (4.3.3) packages. I do not clearly understand what causes this error and how to solve this issue.