Training Keras Model on One-by-One sample and plotting validation curve

user_12980888 0
Viewed 111

I have seen some posts talking about training Keras model on one-by-one sample. I understood from these posts too that Keras model will update the model weights even on one-by-one training basis. So, the question now, in such a case, how we can provide validation split and how we can plot the validation of the model given we are not providing the entire dataset to our model, but row by row of the dataset?

reduce_lr = tf.keras.callbacks.LearningRateScheduler(lambda x: 1e-3 * 0.90 ** x)
model.compile(optimizer=keras.optimizers.Adam(clipnorm=1.0, clipvalue=0.5), loss="mse")
for i in range(train_size):
    one_row = train_data.iloc[i:i+1, :].to_numpy()
    one_row = tf.convert_to_tensor(one_row .reshape(-1, one_row .shape[0], one_row.shape[1]), np.float32)
    one_label=one_row
    history= model.fit(one_row ,one_label, epochs=10, validation_split=0.1,verbose=2, callbacks=[reduce_lr])
keras tensorflow machine-learning
1 Answers

In Keras, you have two options to perform validation:

  1. The first is by specifying a portion of the training data to be used for validation. This portion will be taken from the end or randomly based on the Shuffle parameter passed. This option can be activated by specifying the validation_split:

Float between 0 and 1. Fraction of the training data to be used as validation data. The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch. The validation data is selected from the last samples in the x and y data provided, before shuffling.

  1. The second option is to provide the validation data yourself. You can split your dataset at the very beginning, and take a portion of it (usually 10% to 20%) as a validation data. This can be specified by the validation_data parameter:

Data on which to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data. Thus, note the fact that the validation loss of data provided using validation_split or validation_data is not affected by regularization layers like noise and dropout. validation_data will override validation_split. validation_data could be:

A tuple (x_val, y_val) of Numpy arrays or tensors.
A tuple (x_val, y_val, val_sample_weights) of NumPy arrays.
A tf.data.Dataset.
A Python generator or keras.utils.Sequence returning (inputs, targets) 
or (inputs, targets, sample_weights). validation_data is not
yet supported with tf.distribute.experimental.ParameterServerStrategy.

So you can go with option 2 above, and pass one sample of the validation data per iteration. Yet, you need to define your strategy of how to compensate the difference in the number of samples between the training set and the validation set. You might, for example, reset the index of the validation-set automatically to 0 when it reaches the end of the validation-set.

Regarding how to plot them, this is not quite different from the usual way:

Model.fit() return:

A History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values (if applicable).

Hence, if you have one epoch, extract the training and validation losses / accuracy / metrics ..etc from the History object after every iteration, and save them in corresponding lists. If you have more than one epoch per iteration, then take the average or the final one, based on your requirements..etc.

Finally, at the very end, plot them using, for example, the classic Matplotlib.

user_63651120
Related