How to get f-measure in multiclass-multioutput classification in python?

user_10704050 0
Viewed 837

I have a multiclass-multioutput classification problem where I use randomforest classifier in sklearn to perform the classification. My code is as follows.

from sklearn import datasets
iris = datasets.load_iris()
X = iris.data[:5]
y = np.array([[0,0], [0,1], [0,2], [0,4], [1,1]])
clf = RandomForestClassifier(random_state = 42, class_weight="balanced")
k_fold = StratifiedKFold(n_splits=10, shuffle=True, random_state=42)

I wanted to perform 10-fold cross validation and get the f-measure of the classification. I initially tried the following code.

scores = cross_validate(clf, X, y, cv=k_fold, scoring = ('f1_weighted'))

However, I got the error ValueError: multiclass-multioutput is not supported.

Therefore, I tried the following way by manually looping through the folds in the cross validation.

cv_f = []
for train_index, val_index in k_fold.split(X, y):
    clf.fit(X[train_index], y[train_index])
    pred = clf.predict(X[val_index])
    f = f1_score(y[val_index], pred)
    cv_f.append(f)

However, it did not work too.

It seems like sklearn does not support multiclass-multioutput classification. However, I am pretty sure that there may be other ways of doing this in python.

I am happy to provide more details if needed.

EDIT

I tried the following code.

import numpy as np
from sklearn import datasets
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_validate, KFold
from sklearn.multioutput import MultiOutputClassifier

iris = datasets.load_iris()
X = iris.data[:5]
y = np.array([[0,0], [0,1], [0,2], [0,4], [1,1]])

clf = MultiOutputClassifier(RandomForestClassifier(random_state=42, class_weight="balanced"))
k_fold = KFold(n_splits=3, shuffle=True, random_state=42)

scores = cross_validate(clf, X, y, cv=k_fold, scoring=['f1_weighted'])

However, I stll get the error as raise ValueError("{0} is not supported".format(y_type)) ValueError: multiclass-multioutput is not supported.

Please let me know a suitable way to solve this issue.

scikit-learn classification python
1 Answers

scikit-learn actually does support multiclass-multioutput classification problems. You just need the right module and classifier. Did you know about the sklearn.multioutput module and e.g. the MultiOutputClassifier? With them, you can do:

import numpy as np
from random import choices
from sklearn import datasets
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_validate, KFold
from sklearn.multioutput import MultiOutputClassifier

iris = datasets.load_iris()
X = iris.data[:100]
y = np.array(choices([[0, 0], [0, 1], [1, 0], [1, 1]], k=100))

clf = MultiOutputClassifier(RandomForestClassifier(random_state=42, class_weight="balanced"))
k_fold = KFold(n_splits=5, shuffle=True, random_state=42)

scores = cross_validate(clf, X, y, cv=k_fold, scoring=['f1_weighted'])

This code will work all fine.

user_131828850
Related