I d like to build a Bert model MLM from scratch without using Bert-tockenizer, I have my own vocab (which looks like gibberish: "xddccfff988, 9900zxzxxx, zzzTzz"). At the end, I d like to get embeddings of these gibberish words too.
I v been struggling with this for days and any help is highly appreciated.