Does model get retrained entirely using .fit() in sklearn and tensorflow

user_11278433 0
Viewed 1084

I am trying to use machine learning in Python. Right now I am using sklearn and TensorFlow. I was wondering what to do if I have a model that needs updating when new data comes. For example, I have financial data. I built an LSTM model with TensorFlow and trained it. But new data comes in every day, and I don't want to retrain the model every day. Is there a way just to update the model and not retrain it from scratch?

In sklearn, the documentation for .fit() method (using DecisionTreeClassifier as an example) says that it

Build a decision tree classifier from the training set (X, y).

So it seems like it will retrain the entire model from scratch.

In tensorflow, .fit() method (using Sequential as an example) say

Trains the model for a fixed number of epochs (iterations on a dataset).

So it seems like it does update the model instead of retraining. But I am not sure if my understanding is correct. I would be grateful for some clarification. And if sklearn indeed retrains the entire model using .fit(), is there a function that would just update the model instead of retraining from scratch?

scikit-learn tensorflow python
3 Answers

When you say update and not train. Is it just updating the weights using the new data?

If so you can adopt two approaches with Transfer learning.

  1. Finetune: Initialise a model with the weights from old model and retrain it on new data.
  2. Add a new layer: Add a new layer and update the weights in this layer only while freezing the remaining weights in the network.

for more details read the tensorflow guide on tansferlearning

user_143407690

In tensorflow, there is a method called train_on_batch() that you can call on your model.

Say you defined your model as sequential, and you initially trained it on the existing initial_dataset using the fit method.

Now, you have new data in your hand -> call it X_new_train,y_new_train

so you can update the existing model using train_on_batch()

An example would be:

     #generate some X_new_train (one batch)
     X_new_train = tf.random.normal(shape=[no_of_samples_in_one_batch,100])

     #generate corresponding y_new_train

     y_new_train = tf.constant([[1.0]]*no_of_samples_in_one_batch)

     model.train_on_batch(X_new_train,y_new_train)

Note that the idea of no_of_samples_in_one_batch (also called batch size) is not so important here. I mean whatever number of samples that you have in your data will be considered as one batch!

Now, coming to sklearn, I am not sure whether all machine learning models can incrementally learn (update weights from new examples). There is a list of models that support incremental learning:

https://scikit-learn.org/0.15/modules/scaling_strategies.html#incremental-learning

user_136793600

In sklearn, the .fit() method retrains on the dataset i.e as you use .fit() on any dataset, any info pertaining to previous training will all be discarded. So assuming you have new data coming in every day you will have to retrain each time in the case of most sklearn algorithms. Although, If you like to retrain the sklearn models instead of training from scratch, some algorithms of sklearn (like SGDClassifier) provide a method called partial_fit(). These can be used to retrain and update the weights of an existing model.

As per Tensorflow, the .fit() method actually trains the model without discarding any info pertaining to previous trainings. Hence each time .fit() is used via TF it will actually retrain the model.

Tip: you can use SaveModel from TF to save the best model and reload and re-train the model as and when more data keeps flowing in.

user_112173310
Related