As the documentation of the class is not very clear. I don't understand what value I give it.
cross_val_score(estimator, X, y=None)
This is my code:
clf = LinearSVC(random_state=seed, **params)
cvscore = cross_val_score(clf, features, labels)
I am not sure if this is correct or if I need to give X_train and y_train instead of features and labels.
Thanks
It is always a good idea to seperate the test set and training set, even while using cross_val_score. The reason behind this is knowledge leaking. It basically means that when you use both training and test sets, you are leaking information from test set into your model, thereby making your model biased, leading to incorrect predictions.
Here is detailed blog post on the same issue.
References:
I assume you were referring to the below documentation: sklearn.model_selection.cross_val_score
The purpose of cross validation is ensuring that your model hasn't had particularly high variance leading to a good-fit in one instance, but a poor fit in another instance. This is used generally in model validation. With this in mind, you should be passing the training set (X_train, y_train) and seeing how your model performs.
Your question was focused on: "Can I pass in the whole data-set into cross validation?"
The answer is, yes. This is conditional and based off whether or not you are satisfied with your ML output. Say for example, I have the following below:
I have used a Random forest model and am happy with my overall model fit and score.
In this case, I have a hold-out set. Once I remove this hold-out set and give my model the whole data-set, we would get a plot with an even higher score as I am giving my model more information (and as such, your CV scores will also reflectively be higher).
An example of calling the method might be as such: probablistic_scores = cross_val_score(model, X_train, y_train, cv=5)
Generally a 5 Fold Cross validation is preferred. If you wish to go higher than 5 fold - please note that as your 'n' folds increase, the number of computational resources required will also increase and will take longer to process.