huggingface-tokenizers

115 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Huggingface error: AttributeError: 'ByteLevelBPETokenizer' object has no attribute 'pad_token_id'
user_30236150
•
2
1
1034
huggingface-tokenizers huggingface-transformers pytorch tokenize python
Tokenizing & encoding dataset uses too much RAM
user_148295230
•
2
1
1572
huggingface-tokenizers huggingface-transformers pytorch nlp python
How to map token indices from the SQuAD data to tokens from BERT tokenizer?
user_91511760
•
2
1
2087
squad nlp-question-answering transformer-model huggingface-tokenizers bert-language-model
Applying pre trained facebook/bart-large-cnn for text summarization in python
user_122263770
•
2
1
1979
summarization huggingface-tokenizers huggingface-transformers nlp python-3.x
OSError: Can't load tokenizer
Admin1
•
3
1
3612
transformer-model huggingface-tokenizers python
Enabling truncation in transformers feature extraction pipeline
user_76678870
•
2
0
375
huggingface-tokenizers huggingface-transformers
How to untokenize BERT tokens?
user_79730420
•
4
2
3642
huggingface-tokenizers bert-language-model huggingface-transformers tokenize python
Should I create a PyTorch Dataset to train a model off a pyspark dataframe?
user_42404130
•
3
0
1001
petastorm huggingface-tokenizers pytorch pyspark python
HuggingFace Bert Sentiment analysis
user_56697470
•
1
1
7569
huggingface-tokenizers bert-language-model huggingface-transformers python
Huggingface AlBert tokenizer NoneType error with Colab
user_84261050
•
15
5
6695
huggingface-tokenizers huggingface-transformers google-colaboratory
Hot Questions