Merging 2 data frames in R4.0.0 is much slower than in R.3.6.3.
Reproducible example:
library(reshape2)
n <- matrix(1:6000000, nrow=15000, ncol=400)
colnames(n) <- 1:ncol(n)
meta <- data.frame("col1" = ncol(n):1, row.names = colnames(n))
start_time <- Sys.time()
test = sapply(1:nrow(n),
function(i) {
print(i)
nn = reshape2::melt(n[i,])
tmp = merge(nn, meta, by="row.names");
}
)
end_time <- Sys.time()
end_time-start_time
This code takes 23 seconds in R3.6.3 and 23 minutes in R4.0.0 (on my machine) - making the merge 60 times slower! The 'melt' function is not the time consuming step.
This issue is also not related to sapply. You can reproduce the extreme speed difference using a for loop:
for(i in 1:nrow(n)){
print(i)
nn = reshape2::melt(n[i,])
tmp = merge(nn, meta, by="row.names");
}
I'd be happy to hear your feedback! Am I missing something?
(I have previously asked this question in another context and have deleted the old question since it wasn't reproducible)