What is the difference between validation split and test size?

test_size=0.33 allocates 33% from the whole to dtest, leaving 67% for dtrain.

While calling the fit function, the validation_split splits 20% from the 67% of training data, and used it for validation purpose.

test_size

float or int, default=None If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the test split. If int, represents the absolute number of test samples. If None, the value is set to the complement of the train size. If train_size is also None, it will be set to 0.25.

validation split

Float between 0 and 1. Fraction of the training data to be used as validation data. The model will set apart this fraction of the training data, will not train on it, and will evaluate the loss and any model metrics on this data at the end of each epoch. The validation data is selected from the last samples in the x and y data provided, before shuffling.

Theoritically :

• Hyperparemters: Every machine learning model has hyperparemters that are used to control the learning process (example: learning rate, batch size, weight decay, network depts, network breadth etc). Some combination of these hyperparameters is chosen such that the metrics we are interested in (F1, accuracy etc) are good or above the acceptable level.
• Tuning Hyperparemets: We would like to find the good hyperparemet values. The model is trained using the train data(train split of full data). Since the model have already looked at the ground labels of the train split we cannot use the model on the same split to evaluate the model performance (i.e to evaluate how good our selected hyperparameters are). The model can just memorize (overfit) the train data and the model will perform very good on the train data. However, it will not perform well (generalize) on unseed data.
• Validataion split: So we create a smaller validation split from the full data and use it only for evaluation of the trained model. We repeat the training process with another set of hyperparameters until we find a model whose performance is acceptable over the validation data.
• Generalized model metrics: We have found a best set of hyperparemetrs using the validation split. However, this is not the generalized model performance since we have tuned the hyperparametrs to perform well on the validation split. To identify how well our final set of hyperparmers (model) performs on unseen data, we create a small test split and then evaluate the final trained model on this split. We can expect the same metrics on another unseen data (assuming the unseen data will be from the same distribution) and can be considered as generalized model performance which is expected.

By your code

history = model.fit(
    x=[train_cont_feats, train_cate_feats],
    `y=train_rating`,
    'batch_size=64',
    'epochs=50',
    verbose=1,
    validation_split=.2,

The above code is using the data 90% of (x,y) for training the model and the remaining 20% of (x,y) for evaluation of the trained model per epoch to report the metrics. You use the validation metrics to figure out how well your model is performing. You then will have to repeat this process for a different combination of hyperparameters to figure out a good set. There are also automated ways of doing like grid search, random search, bayes optimization etc.

First of all, you need to know the difference between validation and test set. When we are training supervised machine learning models including deep models we should tune hyperparameters (such as learning rate, number of hidden layers, ...), too. The reason is to find the best configuration of the model. For the purpose, we use the validation set, which is specified in your code by validation_split. We can do greedy, random, Bayesian or other optimization methods to find that. After training and tuning the models we need some intact data to report the unoptimistic results (test_size in your code).