Number of rows assigned for analysis is not equal to the expected number based on proportion specified in initial_split() argument in R

Viewed 66

In R, using the intial_split() function with the default value (which are 0.75 for testing and 0.25 for training), I get the expected number of rows assigned to the testing split. I demonstrate this below with the randu dataset included in R which has 400 rows, so we expect 300 (400 * 0.75) in the training split and 100 (400 * 0.25) in the testing slipt. Here is the code and outputs:

> nrow(randu)
[1] 400
> randu_split <- initial_split(randu)
> randu_train <- training(randu_split)
> randu_test <- testing(randu_split)
> nrow(randu_train)
[1] 300
> nrow(randu_test)
[1] 100

However, when the proportion to attribute to the training and testing are changed to 0.80 and 0.20 using the proportion argument (prop = 4/5), the number of rows assigned to the different split is different than what is expected. In a dataset with 400 rows, I would expect 320 (400 * 0.80) in the training split and 80 (400 * 0.20) in the testing split.

> 400*0.8
[1] 320
> 400*0.2
[1] 80

However, this is not what I observed. Please see results below:

> randu_split_80_20 <- initial_split(randu, prop = 4/5)
> randu_train_eighty <- training(randu_split_80_20)
> randu_test_twenty <- testing(randu_split_80_20)
> nrow(randu_train_eighty)
[1] 321
> nrow(randu_test_twenty)
[1] 79
> nrow(randu_split_80_20)
analysis 
     321 

Why is the intial_split() function with prop = 4/5 assigning 321 and not 320 rows to the training split? I tried this with different datasets and the assignment is always off by one. I also tried with prop = 9/10, and the number of samples returned in the split are also off by one compared to the expected number. Any ideas?

1 Answers

It's probably due to rounding error. The source code of rsample::initial_split can be traced back to mc_cv and mc_splits. And the number of testing is calculated by floor(n * (1 - prop)). To see the numbers in R:

> n = 400 #number of rows
> prop = 3/4
> sprintf("%.20f", n * (1-prop))
[1] "100.00000000000000000000"
> floor(n * (1-prop)) #number of testing set
[1] 100

> prop = 4/5
> sprintf("%.20f", n * (1-prop))
[1] "79.99999999999998578915"
> floor(n * (1-prop))
[1] 79

> prop = 9/10
> sprintf("%.20f", n * (1-prop))
[1] "39.99999999999999289457"
> floor(n * (1-prop))
[1] 39
Related