Compare every group with every other group in data.table

user_4874081 0
Viewed 49

I am looking for a data.table way to potentially eliminate a double for loop. I am trying to compare every group with every other group with the DTW algorithm.

Here a MWE (not optimized):

library(data.table)
library(SimilarityMeasures)

tt <- data.table(A=1:100,B=2:101,C=rep(1:4,each=25))
result <- list()
for(i in 1:4){
  for(j in 1:4){
      result[[4*i+j]] <- DTW(tt[C==i,cbind(A,B)],tt[C==j,cbind(A,B)]) 
  }

}
result

A and B are some variables that are used as the input to the DTW algorithm, C is the grouping variable. I tried to come up with some lapply solution combined with a .SD or .BY approach in data.table, but failed. The goal is to have a much faster function than the above double for loop (even when running it in parallel).

Any help is much appreciated. Thanks!

data.table r loops
1 Answers

I am looking for a data.table way to potentially eliminate a double for loop.

This is my suggestion instead of a for loop and number-indexed list:

s = lapply(split(tt, by="C", keep=FALSE), as.matrix)
f = sum

res = tt[, CJ(C1 = C, C2 = C, unique=TRUE)]
res[, v := f(s[[C1]], s[[C2]]), by=.(C1, C2)]

(I can't install your package and so use some f instead of DTW. I'm converting to matrix, since I assume that's why you use cbind.)

Generally, enumerating combos and using by= is a way to avoid / reformulate a loop with data.table.

Precomputing the subsetted matrices s might help with performance, but this is still essentially a loop. I doubt there is much further optimization that can be made here.

user_11912590
Related