I'm building a random forest model from sklearn and I usually use class_weight="balanced" in my parameters (let's call this model_1).
params={'class_weight': 'balanced',
'max_depth': 10,
'max_features': 'auto',
'min_samples_leaf': 0.25,
'min_samples_split': 0.25,
'n_estimators': 50,
'n_jobs': 6,
'oob_score': True,
'random_state': 21,
'verbose': 1}
model_1 confusion matrix:
-1 0 1 trueTotal
-1 86 67 59 212
0 31 43 32 106
1 74 80 61 215
predTotal 191 190 152 533
I noticed that when I pass in a dictionary calculated from sklearn.utils.class_weight.compute_class_weight using "balanced" my confusion matrix doesn't match model_1's
from sklearn.utils import class_weight
keys = np.unique(tdf[targetName])
values = class_weight.compute_class_weight("balanced", keys, tdf[targetName].values)
cw = dict(zip(keys, values))
cw
{-1: 0.86371284352215372, 0: 1.5802975885069266, 1: 0.82684563758389262}
params1 = params.copy()
params1["class_weight"] = cw
model_2 confusion matrix
-1 0 1 trueTotal
-1 212 0 0 212
0 106 0 0 106
1 215 0 0 215
predTotal 533 0 0 533
I'm really just looking for an example on how I can change the class_weight to try and improve my true predictions for -1 and 1 and penalize false positives for -1 and 1 (0 predictions or 0 actuals for predictions of 1 and -1 are ok) given the following cost matrix. Adjusting the weights aren't producing the predictions that I thought they would so now I'm wondering if my understanding of class_weight is correct.
cost matrix
-1 0 1
-1 10 0 -10
0 0 0 0
1 -10 0 10
** EDIT **
To clarify, class_weight.compute_class_weight("balanced"...) results in the same values as the n_samples / (n_classes * np.bincount(y))
y = np.where(tdf[targetName] == -1, 2, tdf[targetName])
len(y)/(3*np.bincount(y))
array([ 1.58029759, 0.82684564, 0.86371284])
When passed into the model the confusion matrix is not the same as "balanced".