I tried to read the docs for RepeatedStratifiedKFold and StratifiedKFold but couldn't tell the difference between the two methods except that RepeatedStratifiedKFold repeats Stratified K-Fold n times with different randomization in each repetition.
My questions then: Do these two methods return the same thing? Which one should I use to split an imbalanced dataset when doing GridSearchCV and the logic for choosing that method?
A single run of StratifiedKFold might result in a noisy estimate of model performance, as different splits of the data might result in very different results. That's where RepeatedStratifiedKFold comes into play.
RepeatedStratifiedKFold allows improving the estimated performance of a machine learning model, by simply repeating the cross-validation procedure multiple times (according to n_repeats value) and reporting the mean result across all folds from all runs. This mean result is expected to be a more accurate estimate of the model performance (there is a great article here explaining this).
Thus, to answer your question. No, these two methods would not return the same results. As - with RepeatedStratifiedKFold - each time running the procedure results in a different split of the dataset into stratified k-folds, the performance results would be different. RepeatedStratifiedKFold has the benefit of improving the estimated model performance at the cost of fitting and evaluating many more models. If, for example, 5 repeats (n_repeats=5) of 10-fold cross-validation are used for estimating the model performance, this means that 50 different models would need to be fit and evaluated; which might be computationally costly, depending on the dataset's size, type of machine learning algorithm, device specifications, etc. However, RepeatedStratifiedKFold process can be executed on different cores or different machines. For instance, setting n_jobs=-1 would use all available cores (have a look here).
