Error: `data` and `reference` should be factors with the same levels random forrest

Question

This is a code I am doing for an assignment. I cannot seem to get a confusion matrix for the predictions, please assist me with troubleshooting the code or make any recommendations necessary.

set.seed(1234) test_index1 <-createDataPartition(water_potability3$Potability,p=0.1,list= FALSE)

water_potability_train <- water_potability3[test_index1,-c(4,6:9)]

water_potability3_test<- water_potability3[!1:nrow(water_potability3)%in%test_index1,-c(4,6:9)]

<- tuneRF(x=water_potability_train[,1:4],y=water_potability_train$Potability) (mintree <-trf[which.min(trf[,2]),1]) <-randomForest(x=water_potability_train[,-5],y=water_potability_train$Potability,mtry = mintree,importance = TRUE)

(rf_model,main="") (rf_model,main="")

preds_rf<- predict(rf_model,water_potability3_test[,-5])

table(preds_rf,water_potability3_test$Potability)

confusionmatrix(preds_rf,water_potability3_test$Potability)

Everytime I do a confusion matrix I get the error "Error: data and reference should be factors with the same levels"

user_11832955 · Answer

As you don't share a dataset that allows me to reproduce the error, I'm gonna have a guess and provide the solution I would have used myself. If this doesn't work for you, please provide some data and perhaps explain what the Potability column contains :-)

When randomly splitting the data into training and test partitions, you risk not having observations from every class in both partitions. E.g. if you have 10 classes, you might only have 8 of them in the smaller test partition. And then when your model predicts one of the two other classes that were available in the training partition, the two factors have different levels.

So I use partition() from groupdata2 with the cat_col argument that ensures each class is represented in both partitions (if possible). Then I use confusion_matrix() from cvms as it allows different levels in the two factors.

library(groupdata2)
library(cvms)
set.seed(1234) 

# Create list with two partitions
# where the ratio of classes in Potability are similar
parts <- partition(water_potability3[, -c(4,6:9)], 
                   p = 0.1, cat_col = "Potability")

# Extract the two partitions
water_potability3_test <- parts[[1]]
water_potability3_train <- parts[[2]]

# The modeling (haven't changed anything here)
trf <- tuneRF(x = water_potability_train[, 1:4],
              y = water_potability_train$Potability) 

(mintree <- trf[which.min(trf[, 2]), 1]) 

rf_model <- randomForest(
    x = water_potability_train[, -5],
    y = water_potability_train$Potability,
    mtry = mintree,
    importance = TRUE
)

preds_rf <- predict(rf_model, water_potability3_test[, -5])

# Create confusion matrix
conf_mat <- cvms::confusion_matrix(
    targets = water_potability3_test$Potability,
    predictions = preds_rf
)

# The basic confusion matrix table
conf_mat$Table

# Or as a plot
plot_confusion_matrix(conf_mat)

You may also check out cvms::evaluate() that has additional evaluation metrics.

Error: `data` and `reference` should be factors with the same levels random forrest

1 Answers

Read more