R caret: "non-numeric argument to binary operator" in train with qrf

Question

When I run a quantile regression forest model with caret::train, I get the following error: Error in { : task 1 failed - "non-numeric argument to binary operator".

When I set ntree to a higher number (in my reproducible example this would be ntree = 150), my code runs without errors.

This code

library(caret)
library(quantregForest)

data(segmentationData)

dat <- segmentationData[segmentationData$Case == "Train",]
dat <- dat[1:50,]

# predictors
preds <- dat[,c(5:ncol(dat))]

# convert all to numeric
preds <- data.frame(sapply(preds, function(x) as.numeric(as.character(x))))

# response variable
response <- dat[,4]

# set up error measures
sumfct <- function(data, lev = NULL, model = NULL){
  RMSE <- sqrt(mean((data$pred - data$obs)^2, na.omit = TRUE))
  c(RMSE = RMSE)
}


# specify folds
set.seed(42, kind = "Mersenne-Twister", normal.kind = "Inversion")
folds_train <- caret::createMultiFolds(y = dat$Cell,
                                       k = 10,
                                       times = 5)

# specify trainControl for tuning mtry with the created multifolds
finalcontrol <- caret::trainControl(search = "grid", method = "repeatedcv", number = 10, repeats = 5, 
                                    index = folds_train, savePredictions = TRUE, summaryFunction = sumfct)

# build grid for tuning mtry
tunegrid <- expand.grid(mtry = c(2, 10, sqrt(ncol(preds)), ncol(preds)/3))

# train model
set.seed(42, kind = "Mersenne-Twister", normal.kind = "Inversion")
model <- caret::train(x = preds, 
                      y = response,
                      method ="qrf",
                      ntree = 30, # with ntree = 150 it works
                      metric = "RMSE",
                      tuneGrid = tunegrid,
                      trControl = finalcontrol,
                      importance = TRUE,
                      keep.inbag = TRUE
)

produces the error. The model with my real data has ntree = 10000 and still the task is failing. How can I fix this?

Where in the source code of caret can I find the conditions for the error message Error in { : task 1 failed - "non-numeric argument to binary operator"? From which part of the source code does the error message come from?

UPDATE: I adapted my code with my real data according to the answer of StupidWolf, so it looks like this:

# train model
set.seed(42, kind = "Mersenne-Twister", normal.kind = "Inversion")
model <- caret::train(x = preds, 
                      y = response,
                      method ="qrf",
                      ntree = 30, # with ntree = 150 it works
                      metric = "RMSE",
                      sampsize = ceiling(length(response)*0.4)
                      tuneGrid = tunegrid,
                      trControl = finalcontrol,
                      importance = TRUE,
                      keep.inbag = FALSE
)

With my real data I still get the above error message, so that I had to adapt the sampsize to 0.1*length(response) in the worst case in order to compute the model successfully. So only setting keep.inbag = FALSEstill produced errors. I have up to 1500 predictors while the number of samples (rows) are only 50 to 60. I still don't understand, what exactly causes the error message. I tried the model without the sampsize argument, but always set keep.inbag = FALSE. The error was still occuring. only setting the sampsize very low ensured success.

How can I run the model successfully without setting sampsize? I actually wanted the bootstrap approach for the out of bag data sets and not the artificial sampsize of 40 % or 10% of my data set for training the forest.

user_12258459 · Answer

You get the error because you used the option keep.inbag = TRUE, in the quantregforest code, line 95:

minoob <- min( apply(!is.na(valuesPredict),1,sum))
if(minoob<10) stop("need to increase number of trees for sufficiently many out-of-bag observations")

So it requires that all of your observations have at least 10 instances of OOB (out of bag), to keep the out of bag predictions. So if your real data is huge, the ntrees required for keeping the out of bag is going to be huge.

If you are using caret for training the data, keeping the OOB and having savePredictions = TRUE seems redundant. On the whole, OOB predictions might not be so useful since you will be using the test fold to predict anyway.

Another option, given the size of your data, is to tweak the sampsize. In randomForest only a number of sampsize observations are sampled with replacement subset to fit a tree. If you set a lower size for this, you ensure there's enough OOB. For example in the example given, we can see:

model <- caret::train(x = preds, 
                      y = response,
                      method ="qrf",
                      ntree = 30, sampsize=17,
                      metric = "RMSE",
                      tuneGrid = tunegrid,
                      trControl = finalcontrol,
                      importance = TRUE,
                      keep.inbag = TRUE)

model
Quantile Random Forest 

50 samples
57 predictors

No pre-processing
Resampling: Cross-Validated (10 fold, repeated 5 times) 
Summary of sample sizes: 44, 43, 44, 46, 45, 46, ... 
Resampling results across tuning parameters:

  mtry       RMSE    
   2.000000  42.53061
   7.549834  42.72116
  10.000000  43.11533
  19.000000  42.80340

RMSE was used to select the optimal model using the smallest value.
The final value used for the model was mtry = 2.

R caret: "non-numeric argument to binary operator" in train with qrf

1 Answers