huggingface-tokenizers

115 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Using huggingface library gives an error: KeyError: 'logits'
user_147354510
•
1
1
1010
huggingface-tokenizers huggingface-transformers pytorch python
How padding in huggingface tokenizer works?
user_63579160
•
3
1
2882
transformer-model huggingface-tokenizers bert-language-model huggingface-transformers nlp
transformers AutoTokenizer.tokenize introducing extra characters
user_97514110
•
4
1
791
huggingface-tokenizers huggingface-transformers python
Adding 'decoder_start_token_id' with SimpleTransformers
user_74580180
•
2
0
412
simpletransformers seq2seq huggingface-tokenizers huggingface-transformers python
Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation
user_122347530
•
2
1
3267
huggingface-tokenizers huggingface-transformers
Huggingface Tokenizer object is not callable
user_35265550
•
2
1
1831
huggingface-tokenizers
Tokenizers change vocabulary entry
user_91024370
•
3
2
600
huggingface-tokenizers huggingface-transformers nlp python-3.x python
Is there a tokenizer that can find sentence boundaries and apply BPE at the same time?
user_21536360
•
2
0
135
sentence huggingface-tokenizers huggingface-transformers tokenize nlp
Suppress HuggingFace logging warning: "Setting `pad_token_id` to `eos_token_id`:{eos_token_id} for open-end generation."
user_45704720
•
4
1
2457
huggingface-tokenizers huggingface-transformers
AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'to_tensor'
user_171541350
•
2
1
1227
huggingface-tokenizers huggingface-transformers google-colaboratory tensorflow python
Hot Questions