Average of predictions from many models versus prediction from the best fitted model?

user_1936401 0
Viewed 60

I have a dataset containing about 700 data points. I randomly shuffle the dataset, then split the dataset in 80:20 ratio for training and testing. Then I use Gaussian process regression (GPR) for fitting the training dataset.

I repeated this process 6 times, and each time I get a new model with varying R2 between 0.50 to 0.80. Now which model should I use for further predictions? Should I take an average of predictions from all the six models, or just choose the best fitted model? If I chose the best fitted model, will there be concern for reproducibility? If there is any other better approach for the fitting, please let me know. Thanks.

gaussian-process scikit-learn regression machine-learning validation
1 Answers

You can try a Gaussian mixture model that combines all your GP models and gives you a probability distribution over all your results. Also, you can provide different weights for each GP model you have when forming a Gaussian mixture. The weights could be your R2 for example.

user_103083800
Related