Merging data frames in R4.0.0 60x slower than R3.6.3?

Viewed 75

Merging 2 data frames in R4.0.0 is much slower than in R.3.6.3.

Reproducible example:

library(reshape2)

n <- matrix(1:6000000, nrow=15000, ncol=400)
colnames(n) <- 1:ncol(n)

meta <- data.frame("col1" = ncol(n):1, row.names = colnames(n))
start_time <- Sys.time()
test = sapply(1:nrow(n), 
               function(i) {
                 print(i)
                 nn = reshape2::melt(n[i,])
                 tmp = merge(nn, meta, by="row.names");
               }
)
end_time <- Sys.time()
end_time-start_time

This code takes 23 seconds in R3.6.3 and 23 minutes in R4.0.0 (on my machine) - making the merge 60 times slower! The 'melt' function is not the time consuming step.

This issue is also not related to sapply. You can reproduce the extreme speed difference using a for loop:

for(i in 1:nrow(n)){
  print(i)
  nn = reshape2::melt(n[i,])
  tmp = merge(nn, meta, by="row.names");
}

I'd be happy to hear your feedback! Am I missing something?

(I have previously asked this question in another context and have deleted the old question since it wasn't reproducible)

1 Answers

This is actually a bug in R4.0.0.

I have submitted it to R-core (https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17794) and update here once it is fixed.

Update The bug has been fixed by the R-team (thanks to Martin Maechler!), is already rolled out in R-devel, and the fix will be included in the next release version of R.

Related