dropout(): argument 'input' (position 1) must be Tensor, not str when using Bert with Huggingface

user_14738280 0
Viewed 20681

My code was working fine and when I tried to run it today without changing anything I got the following error:

dropout(): argument 'input' (position 1) must be Tensor, not str

Would appreciate if help could be provided.
Could be an issue with the data loader?

bert-language-model pytorch
3 Answers

if you use HuggingFace, this information could be useful. I have same error and fix it with adding parameter return_dict=False in model class before dropout: outputs = model(**inputs, return_dict=False)

user_129877050

I was also working on same repo. There is a class probably named Bert_Arch that inherits the nn.Module and this class has a overriden method named forward. Inside forward method just add the parameter 'return_dict=False' to the self.bert() method call. Replace

_, cls_hs = self.bert(sent_id, attention_mask=mask)

with

_, cls_hs = self.bert(sent_id, attention_mask=mask, return_dict=False)
user_62197570

If you are using the Hugging Face transformers library, this error pops up when running code written in v3 on the transformers v4 library. To resolve it, simply add return_dict=False when loading the model as follows:

model = BertModel.from_pretrained("bert-base-cased")
outputs = model(**inputs, return_dict=False)

or

model = BertModel.from_pretrained("bert-base-cased", return_dict=False)
outputs = model(**inputs)

I hope this helps. It worked for me.

Reference: https://huggingface.co/transformers/migration.html

user_127235620
Related