Retraining existing base BERT model with additional data

user_1872999 0
Viewed 1066

I have generated new Base BERT model(dataset1_model_cased_L-12_H-768_A-12) using cased_L-12_H-768_A-12 as trained multi label classification from biobert-run_classifier

I need to add more additional data as dataset2 and the model should be dataset2_model_cased_L-12_H-768_A-12

Is tensorflow-hub help this to resolve my problem?

Model training life cycle will be like this below,

cased_L-12_H-768_A-12 => dataset1 => dataset1_model_cased_L-12_H-768_A-12

dataset1_model_cased_L-12_H-768_A-12 => dataset2 => dataset2_model_cased_L-12_H-768_A-12

tensorflow-hub bert-language-model tensorflow
1 Answers

Tensorflow Hub is a platform for sharing pre-trained model pieces or whole models, and an API to facilitate this sharing. In TF 1.x, this API was a stand-alone API and in TF 2.x this API (SavedModel: https://www.tensorflow.org/guide/saved_model) is part of the core TF API.

In the proposed training life-cycle example, using SavedModel to save relevant model between the training steps could simplify pipeline architecture design. Alternatively, you could use coding examples available as part of the TF Model Garden to perform this pre-training: https://github.com/tensorflow/models/tree/master/official/nlp.

user_96908750
Related