I'm having a project for ner, and i want to use pipline component of spacy for ner with word vector generated from a pre-trained model in the transformer. I using spacy-transformer of spacy and follow their guild but it not work.
I'm using spacy-2.3.5, transformer-0.6.2, python-2.3.5 and trying to run it in colab.
Link of spacy-transformer GitHub: Link to git hub
Link to the model in the transformer: Link to vinai/phobert-base
Name of model in tranform: vinai/phobert-base
I have a question: Whether we can use any pre-train model in the transformer via spacy-transformer or just some kind of model?
In their guild, before loading pre-trained model in spacy we need to initialize it. here their guild
! export CUDA_PATH="/opt/nvidia/cuda"
! pip install -U spacy[cuda101]
! pip install spacy-transformers
! git clone -b v0.6.x https://github.com/explosion/spacy-transformers
! python /content/spacy-transformers/examples/init_model.py -n "vinai/phobert-base" \
-l vi vi_vinai_phobert_base
I received a log:
2020-12-25 03:46:18.163785: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
ℹ Creating model for 'vinai/phobert-base' (vi)
Downloading: 100% 557/557 [00:00<00:00, 689kB/s]
⠼ Setting up the pipeline...
Traceback (most recent call last):
File "/content/spacy-transformers/examples/init_model.py", line 32, in <module>
plac.call(main)
File "/usr/local/lib/python3.6/dist-packages/plac_core.py", line 367, in call
cmd, result = parser.consume(arglist)
File "/usr/local/lib/python3.6/dist-packages/plac_core.py", line 232, in consume
return cmd, self.func(*(args + varargs + extraopts), **kwargs)
File "/content/spacy-transformers/examples/init_model.py", line 19, in main
nlp.add_pipe(TransformersWordPiecer.from_pretrained(nlp.vocab, name))
File "/usr/local/lib/python3.6/dist-packages/spacy_transformers/pipeline/wordpiecer.py", line 26, in from_pretrained
model = get_tokenizer(trf_name).from_pretrained(trf_name)
File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py", line 393, in from_pretrained
return cls._from_pretrained(*inputs, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py", line 496, in _from_pretrained
list(cls.vocab_files_names.values()),
OSError: Model name 'vinai/phobert-base' was not found in tokenizers model name list (roberta-base, roberta-large, roberta-large-mnli, distilroberta-base, roberta-base-openai-detector, roberta-large-openai-detector). We assumed 'vinai/phobert-base' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.json', 'merges.txt'] but couldn't find such vocabulary files at this path or url.
Anyone can help me out or give a give me advice? Thank you so much !