tokenize

157 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
What does merge.txt file mean in BERT-based models in HuggingFace library?
user_134943870
•
4
1
923
bert-language-model huggingface-transformers tokenize nlp
How to stop BERT from breaking apart specific words into word-piece
user_54062940
•
6
3
3238
bert-language-model tokenize nlp text python
How nltk.TweetTokenizer different from nltk.word_tokenize?
user_84221700
•
8
1
8465
artificial-intelligence tokenize nltk nlp python
How to choose num_words parameter for keras Tokenizer?
user_79818210
•
2
1
1619
keras tensorflow machine-learning tokenize nlp
How to ignore punctuation in-between words using word_tokenize in NLTK?
user_98555880
•
3
2
1700
tokenize nltk nlp python
About get_special_tokens_mask in huggingface-transformers
user_110561720
•
4
1
1098
huggingface-transformers tokenize
Split the sentence into its tokens as a character annotation Python
user_110593110
•
3
1
976
python-re tokenize nltk python-3.x python
BERT training with character embeddings
user_37419510
•
5
2
2097
transformer-model pytorch tokenize nlp
How to reconstruct text entities with Hugging Face's transformers pipelines without IOB tags?
user_25730690
•
7
3
7386
named-entity-recognition transformer-model huggingface-transformers tokenize nlp
Extracting words from a string into dynamic 2D char array
user_49551840
•
2
1
34
dynamic-arrays multidimensional-array tokenize c string
Hot Questions