How to build spacy spancat model using spacy blank model?

Viewed 20

I have annotated my data in a pre DocBin format:

input = [('Just for me', [(0, 11, 'label_1')]),
     ('For my team', [(0, 11, 'label_1')]),
     ('Organization', [(0, 12, 'label_2')]) and so on....]

Now, I am running the well known formatter to spacy DocBin format:

nlp = spacy.blank('en')
# Create a DocBin object:
db = DocBin()
for text, annotations in input: # Data in previous format
    doc = nlp(text)
    ents = []
    for start, end, label in annotations: # Add character indexes
        span = doc.char_span(start, end, label=label)
        ents.append(span)
    doc.ents = ents # Label the text with the ents
    db.add(doc)
db.to_disk('./train_spacy') 

Now I would like to train a spancat model but I am getting:

ValueError: [E143] Labels for component 'spancat' not initialized. This can be fixed by calling add_label, or by providing a representative batch of examples to the component's 'initialize' method.

When trying to train a model:

!python -m spacy init fill-config base_config.cfg config.cfg
!python -m spacy train config.cfg --output ./output --paths.train ./train.spacy --paths.dev ./train.spacy

Where the base config is taken out from the spacy docs:

# This is an auto-generated partial config. To use it with 'spacy train'
# you can run spacy init fill-config to auto-fill all default settings:
# python -m spacy init fill-config ./base_config.cfg ./config.cfg
[paths]
train = null
dev = null
vectors = null
[system]
gpu_allocator = null
seed = 444
[nlp]
lang = "en"
pipeline = ["tok2vec","spancat"]
batch_size = 1000

[components]

[components.tok2vec]
factory = "tok2vec"

[components.tok2vec.model]
@architectures = "spacy.Tok2Vec.v2"

[components.tok2vec.model.embed]
@architectures = "spacy.MultiHashEmbed.v2"
width = ${components.tok2vec.model.encode.width}
attrs = ["NORM", "PREFIX", "SUFFIX", "SHAPE"]
rows = [5000, 1000, 2500, 2500]
include_static_vectors = true

[components.tok2vec.model.encode]
@architectures = "spacy.MaxoutWindowEncoder.v2"
width = 256
depth = 8
window_size = 1
maxout_pieces = 3

[components.spancat]
factory = "spancat"
max_positive = null
scorer = {"@scorers":"spacy.spancat_scorer.v1"}
spans_key = "sc"
threshold = 0.5

[components.spancat.model]
@architectures = "spacy.SpanCategorizer.v1"

[components.spancat.model.reducer]
@layers = "spacy.mean_max_reducer.v1"
hidden_size = 128

[components.spancat.model.scorer]
@layers = "spacy.LinearLogistic.v1"
nO = null
nI = null

[components.spancat.model.tok2vec]
@architectures = "spacy.Tok2VecListener.v1"
width = ${components.tok2vec.model.encode.width}

[components.spancat.suggester]
@misc = "spacy.ngram_suggester.v1"
sizes = [1,2,3]

[corpora]

[corpora.train]
@readers = "spacy.Corpus.v1"
path = ${paths.train}
max_length = 0

[corpora.dev]
@readers = "spacy.Corpus.v1"
path = ${paths.dev}
max_length = 0

[training]
dev_corpus = "corpora.dev"
train_corpus = "corpora.train"

[training.optimizer]
@optimizers = "Adam.v1"

[training.batcher]
@batchers = "spacy.batch_by_words.v1"
discard_oversize = false
tolerance = 0.2

[training.batcher.size]
@schedules = "compounding.v1"
start = 100
stop = 1000
compound = 1.001

[initialize]
vectors = ${paths.vectors}

Please advise how can run it properly, there must be an example how to do it from A-Z.

0 Answers
Related