SpaCy 3 Cross validation

user_15886390 0
Viewed 532

I was guessing whether there are some build-in options using SpaCy 3 in combination with k-folds cross-validation.

I would like to run a 10 folds cross validation on a dataset of approx. 17.5k job descriptions to receive the best performing model in the end.

Documentation isn't telling anything about these options.

I think I can do some workaround by hand if the option does not exist.

Thanks in advance!

named-entity-recognition spacy python
1 Answers

spaCy does not have any built-in handling for cross validation, no.

With v3 using the config training, you'd probably want to generate train/dev split .spacy files before several training runs to do cross validation.

user_3557150
Related