KeyError when using non-default models in Huggingface transformers pipeline

user_13719811 0
Viewed 1955

I have no problems using the default model in the sentiment analysis pipeline. 

# Allocate a pipeline for sentiment-analysis
nlp = pipeline('sentiment-analysis')

nlp('I am a black man.')

>>>[{'label': 'NEGATIVE', 'score': 0.5723695158958435}]

But, when I try to customise the pipeline a little by adding a specific model. It throws a KeyError. 

nlp = pipeline('sentiment-analysis',
               tokenizer = AutoTokenizer.from_pretrained("DeepPavlov/bert-base-cased-conversational"),
               model = AutoModelWithLMHead.from_pretrained("DeepPavlov/bert-base-cased-conversational"))

nlp('I am a black man.')



>>>---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-55-af7e46d6c6c9> in <module>
      3                tokenizer = AutoTokenizer.from_pretrained("DeepPavlov/bert-base-cased-conversational"),
      4             model = AutoModelWithLMHead.from_pretrained("DeepPavlov/bert-base-cased-conversational"))
----> 5 nlp('I am a black man.')
      6 
      7 

~/opt/anaconda3/lib/python3.7/site-packages/transformers/pipelines.py in __call__(self, *args, **kwargs)
    721         outputs = super().__call__(*args, **kwargs)
    722         scores = np.exp(outputs) / np.exp(outputs).sum(-1, keepdims=True)
--> 723         return [{"label": self.model.config.id2label[item.argmax()], "score": item.max().item()} for item in scores]
    724 
    725 

~/opt/anaconda3/lib/python3.7/site-packages/transformers/pipelines.py in <listcomp>(.0)
    721         outputs = super().__call__(*args, **kwargs)
    722         scores = np.exp(outputs) / np.exp(outputs).sum(-1, keepdims=True)
--> 723         return [{"label": self.model.config.id2label[item.argmax()], "score": item.max().item()} for item in scores]
    724 
    725 

KeyError: 58129
huggingface-transformers
1 Answers

I am facing the same problem. I am working with a model from XML-R fine-tuned with squadv2 data set ("a-ware/xlmroberta-squadv2"). In my case, the KeyError is 16.

Link

Looking for help on the issue I have found this information: link I hope you find it helpful.

Answer (from the link)

The pipeline throws an exception when the model predicts a token that is not part of the document (e.g. final special token [SEP])

My problem:

from transformers import XLMRobertaTokenizer, XLMRobertaForQuestionAnswering
from transformers import pipeline

nlp = pipeline('question-answering', 
                model =  XLMRobertaForQuestionAnswering.from_pretrained('a-ware/xlmroberta-squadv2'),
                tokenizer= XLMRobertaTokenizer.from_pretrained('a-ware/xlmroberta-squadv2'))

nlp(question = "Who was Jim Henson?", context ="Jim Henson was a nice puppet")

---------------------------------------------------------------------------

KeyError                                  Traceback (most recent call last)

<ipython-input-15-b5a8ece5e525> in <module>()
      1 context = "Jim Henson was a nice puppet"
      2 # --------------- CON INTERROGACIONES
----> 3 nlp(question = "Who was Jim Henson?", context =context)

1 frames

/usr/local/lib/python3.6/dist-packages/transformers/pipelines.py in <listcomp>(.0)
   1745                         ),
   1746                     }
-> 1747                     for s, e, score in zip(starts, ends, scores)
   1748                 ]
   1749 

KeyError: 16

Solution 1: Adding punctuation at the end of the context

In order to avoid the bug of trying to extract the final token (which may be an special one as [SEP]) I added an element (in this case a punctuation mark) at the end of the context:

nlp(question = "Who was Jim Henson?", context ="Jim Henson was a nice puppet.")

[OUT]
{'answer': 'nice puppet.', 'end': 28, 'score': 0.5742837190628052, 'start': 17}

Solution 2: Do not use pipeline()

The original model can handle itself to retrieve the correct token`s index.

from transformers import XLMRobertaTokenizer, XLMRobertaForQuestionAnswering
import torch

tokenizer = XLMRobertaTokenizer.from_pretrained('a-ware/xlmroberta-squadv2')
model = XLMRobertaForQuestionAnswering.from_pretrained('a-ware/xlmroberta-squadv2')

question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet"
encoding = tokenizer(question, text, return_tensors='pt')
input_ids = encoding['input_ids']
attention_mask = encoding['attention_mask']

start_scores, end_scores = model(input_ids, attention_mask=attention_mask, output_attentions=False)[:2]

all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0])
answer = ' '.join(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1])
answer = tokenizer.convert_tokens_to_ids(answer.split())
answer = tokenizer.decode(answer)

Update

Looking in more detail your case, I found that the default model for Conversational task in the pipeline is distilbert-base-cased (source code).

The first solution I posted is not a good solution indeed. Trying other questions I got the same error. However, the model itself outside the pipeline works fine (as I showed in solution 2). Thus, I believe that not all models can be introduced in the pipeline. If anyone has more information about it please help us out. Thanks.

user_131805000
Related