My problem is whether I should use TimeSeriesSplit for cross validation of my classifier models (SVC, Random Forest etc).
The input data has a bit of temporal sense. The X features are rolling(1yr, 2yr or whatsoever) z-score of daily market data. So the raw data is time series data, but the data to be learned by Classifier models are in z_score format.
On the other hand, the target feature y represents whether a return of one investment asset exceeds the other after certain period of time(1 week, 1 month or so)
In short, I'm trying to figure out if the z_scores of today's market data(which are historically rolling) have certain impact on the future return of two different invesetment assets.
so my input data will basically look something like the following table:
| y_target | X1 | X2 | X3 |
|---|---|---|---|
| 1 | 1.8 | 1.0 | -1.9 |
| 0 | 1.6 | 1.0 | -2.0 |
| 0 | 1.6 | 0.8 | -1.9 |
| 0 | 1.7 | 0.6 | -1.8 |
| 1 | 1.7 | 0.3 | -1.8 |
| 1 | 1.8 | 0.4 | -1.7 |
| 1 | 1.5 | 0.1 | -2.1 |
| 0 | 1.4 | -0.1 | -2.2 |
| 0 | 1.3 | -0.2 | -2.1 |
| 0 | 1.4 | 0.1 | -2.2 |
| 1 | 1.3 | -0.4 | -2.3 |
| 1 | 1.2 | -0.5 | -2.2 |
And here I'm wondering how I should apply TimeSeriesSplit for validation of my model, because I was taught that Time series data must follow certain temporal validation methodology.
If I'm just wanting to grasp the relative strength between two investment asset as of today, using the z_score level of X_features, will it be fine to just use random cross validation?
The codes that I'm using right now look like this:
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, random_state=123)
param_grid=[{'kernel': ['rbf'], 'C': [0.1, 1, 10],
'gamma': [0.1, 1, 10]},
{'kernel': ['linear'], 'C': [0.1, 1, 10],
'gamma': [0.1, 1, 10]}]
grid_svm=GridSearchCV(SVC(random_state = 123, probability=True), param_grid, cv=5)
grid_svm.fit(X_train, y_train)
y_svm = grid_svm.predict(X_test)
or
rcf = RandomForestClassifier(n_estimators=500, max_depth = None, oob_score=True, random_state=123)
rcf.fit(X_train, y_train)
y_rf = rcf.predict(X_test)
rcf.oob_score_
I want to know if the validation process above has severe theoretical harm in data analysis.
If I must use some kind of forward-testing method for Classifiers, there are thousands of rows of the data and I just don't know how to apply TimeSeriesSplit analysis for such data in this code.