I have 3 time points, considered T=0h, T=0.5H, and T=24H. I trained a RFC model on data at T=24H, to classify two conditions, let's say condition A and condition B. Knowing that the data I am looking at evolves in time, it is expected that my model will not be able to classify data that come from T=0H and T=0.5H.
My model returns testing score at 0.97 and a 10-fold CV score at 0.97 when tested on data from T=24H.
The problem is, when I test this same model (trained on T=24H) on T=0 and T=0.5H, I have low testing scores as expected (0.5 and 0.6) but I have much higher KCV scores (0.9 and 0.8) when I'm expecting scores around 0.5.
Is there a suitable explanation for this?