I am trying to understand what merge.txt file infers in tokenizers for RoBERTa model in HuggingFace library. However, nothing is said about it on their website. Any help is appreciated.
I am trying to understand what merge.txt file infers in tokenizers for RoBERTa model in HuggingFace library. However, nothing is said about it on their website. Any help is appreciated.
You can find a description here:
https://github.com/huggingface/transformers/issues/4777
https://github.com/huggingface/transformers/issues/1083#issuecomment-524303077