Whitelist tokens for text generation (XLNet, GPT-2) in huggingface-transformers

Viewed 232

In the documentation on text generation (https://huggingface.co/transformers/main_classes/model.html#generative-models) there is the option to put

bad_words_ids (List[int], optional) – List of token ids that are not allowed to be generated. In order to get the tokens of the words that should not appear in the generated text, use tokenizer.encode(bad_word, add_prefix_space=True).

Is there also the option to put something along the lines of "allowed_words_ids"? The idea would be to restrict the language of the generated texts.

0 Answers
Related