How to convert from CoNLL format to spacy format

Viewed 1521

I'm currently working on a NER model. I have a bunch of data stored in CoNLL format that need to be converted to Spacy format. In CoNLL, each word of a sentence has a tag next to it. In Spacy, the tag is only shown to the words that have an actually tag. How can I convert from this format below (CoNLL)

From    O
2001    B-DateTime
to  I-DateTime
2004    I-DateTime
,   O
I   O
was O
a   O
stagehand   O
for O
Hartford    B-Company
Stage   I-Company
Company O
.   O

to this format below (Spacy)

TRAIN_DATA = [('what is the price of polo?', {'entities': [(21, 25, 'PrdName')]}), 
              ('what is the price of ball?', {'entities': [(21, 25, 'PrdName')]}), 
              ('what is the price of jegging?', {'entities': [(21, 28, 'PrdName')]}), 
              ('what is the price of t-shirt?', {'entities': [(21, 28, 'PrdName')]}), 
              ('what is the price of jeans?', {'entities': [(21, 26, 'PrdName')]}), 
              ('what is the price of bat?', {'entities': [(21, 24, 'PrdName')]}), 
              ('what is the price of shirt?', {'entities': [(21, 26, 'PrdName')]}), 
              ('what is the price of bag?', {'entities': [(21, 24, 'PrdName')]}), 
              ('what is the price of cup?', {'entities': [(21, 24, 'PrdName')]}), 
              ('what is the price of jug?', {'entities': [(21, 24, 'PrdName')]}), 
              ('what is the price of plate?', {'entities': [(21, 26, 'PrdName')]}), 
              ('what is the price of glass?', {'entities': [(21, 26, 'PrdName')]}),
              ('what is the price of watch?', {'entities': [(21, 26, 'PrdName')]})]
1 Answers

Just use spacy convert.

spacy convert input.conll -c conll ./output/

Note that by default this produces a binary .spacy file. The JSON format is deprecated in v3 and isn't really helpful for much.

Related