Avoid failing when a factor has new levels in test set

Viewed 5727

I have a dataset, which I am splitting into train and test subsets in the following way:

train_ind <- sample(seq_len(nrow(dataset)), size=(2/3)*nrow(dataset))
train <- dataset[train_ind]
test <- dataset[-train_ind]

Then, I use it to train a glm:

glm.res <- glm(response ~ ., data=dataset, subset=train_ind, family = binomial(link=logit)) 

And finally, I use it to predict on my test set:

preds <- predict(glm.res, test, type="response")

Depending on the sample, this fails with error:

Error in model.frame.default(Terms, newdata, na.action = na.action, xlev = object$xlevels) : factor has new levels

Note that the value appears on the full dataset, but apparently not on the training set. What I want to do is make the predict function ignore these new factors. Even if it has performed binarization for the factors, I don't see why it can assume that new values (thus, not variables in the linear model) are simply 0, that would yield the correct behaviour.

Is there a way to do this?

1 Answers
Related