huggingface-tokenizers

115 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
How to use threads for huggingface transformers
user_71645810
•
8
0
1877
huggingface-tokenizers huggingface-transformers threadpool multithreading python
XLNetTokenizer requires the SentencePiece library but it was not found in your environment
user_80687330
•
4
2
4476
transformer-model huggingface-tokenizers huggingface-transformers google-colaboratory
Transformers v4.x: Convert slow tokenizer to fast tokenizer
user_124833460
•
17
3
14584
huggingface-tokenizers huggingface-transformers nlp python
Download pre-trained sentence-transformers model locally
user_99399780
•
8
2
9648
sentence-transformers huggingface-tokenizers bert-language-model word-embedding
Huggingface MarianMT translators lose content, depending on the model
user_48759260
•
3
1
524
huggingface-tokenizers huggingface-transformers machine-translation
what 's the meaning of "Using bos_token, but it is not set yet."
user_47929640
•
6
2
2966
distilbert huggingface-tokenizers huggingface-transformers multilingual
How does max_length, padding and truncation arguments work in HuggingFace' BertTokenizerFast.from_pretrained('bert-base-uncased') work??
user_117250560
•
15
1
18731
huggingface-tokenizers bert-language-model pytorch deep-learning python
On-the-fly tokenization with datasets, tokenizers, and torch Datasets and Dataloaders
user_98247680
•
4
1
1535
gpt-2 huggingface-tokenizers huggingface-transformers
BertModel transformers outputs string instead of tensor
user_7107340
•
11
2
5681
huggingface-tokenizers bert-language-model huggingface-transformers
Finetune TFBertForMaskedLM model.fit() ValueError
user_146302020
•
2
0
305
huggingface-tokenizers tensorflow2.0 huggingface-transformers
Hot Questions