Tensorflow NLP with BERT Preprocessing data

Viewed 48

so this is a specific question involving two Tensorflow text classification tutorials on tensorflow.org. Sorry if this is the wrong place to ask.

Basically, there are two tutorials, one is "Classify Text with BERT" https://www.tensorflow.org/text/tutorials/classify_text_with_bert

And the other is "Fine-tuning a BERT model" https://www.tensorflow.org/text/tutorials/fine_tune_bert

In these two tutorials, it describes preprocessing data. In "Classify Text with BERT", they use a preprocessing model provided by Tensorflow Hub, but in "Fine-tuning a BERT model", they implement python code which tokenizes the data and encodes it and some other stuff. Basically, it seems like the latter method is a lot more complicated than the former.

My question is, why does one tutorial use a preprocessing model provided, while the other actually implements python code? Is there a difference between the two tutorials that requires them to use their specific preprocessing methods?

Thank you!

0 Answers
Related