Top 10 features SVC with rbf kernel

user_7245549 0
Viewed 1349

I'm trying to get the top 10 most informative (best) features for a SVM classifier with RBF kernel. As I'm a beginner in programming, I tried some codes that I found online. Unfortunately, none work. I always get the error: ValueError: coef_ is only available when using a linear kernel.

This is the last code I tested:

scaler = StandardScaler(with_mean=False)
enc = LabelEncoder()
y = enc.fit_transform(labels)
vec = DictVectorizer()

feat_sel = SelectKBest(mutual_info_classif, k=200)

# Pipeline for SVM classifier
clf = SVC()
pipe = Pipeline([('vectorizer', vec),
             ('scaler', StandardScaler(with_mean=False)),
             ('mutual_info', feat_sel),
             ('svc', clf)])


y_pred = model_selection.cross_val_predict(pipe, instances, y, cv=10)


# Now fit the pipeline using your data
pipe.fit(instances, y)

def show_most_informative_features(vec, clf, n=10):
    feature_names = vec.get_feature_names()
    coefs_with_fns = sorted(zip(clf.coef_[0], feature_names))
    top = zip(coefs_with_fns[:n], coefs_with_fns[:-(n + 1):-1])
    for (coef_1, fn_1), (coef_2, fn_2) in top:
        return ('\t%.4f\t%-15s\t\t%.4f\t%-15s' % (coef_1, fn_1, coef_2, fn_2))
print(show_most_informative_features(vec, clf))

Does someone no a way to get the top 10 features from a classifier with a RBF kernel? Or another way to visualize the best features?

feature-selection scikit-learn svm python
2 Answers

There are at least two options available for feature selection for an SVM classifier with RBF kernel within the scikit-learn Python module

  1. If you are performing univariate classification, you can use SelectKBest For classification feature selection, one of the following scoring functions can be specified within SelectKBest :
  1. For sparse data sets, the SequentialFeatureSelector can progressively add or remove features (forward or backward selection) based on the cross-validation score for the classifier. While SFS does not require the classifier model to expose a "coef_" or "feature_importances_" attribute, it may be slow as it will require running m*k fits (i.e., adding/removing m features with k-fold cross-validation).

A recursive feature elimination (RFE) feature selection approach has also been proposed Liu et.al. in Feature selection for support vector machines with RBF kernel.

user_109804140
Related