Why the new columns created with := and tstrsplit are different with or without `by` argument?

Viewed 89

See the following piece of code. I was expecting DT2 to be identical with DT1 after new columns were created. But they are not. What's the possible reason?

library(data.table)

DT1 <- data.table(col1 = c('a', 'a_b'))
DT2 <- copy(DT1)

DT1[, c('col_a', 'col_b') := tstrsplit(col1, '_')]
DT2[, c('col_a', 'col_b') := tstrsplit(col1, '_'), by = .(col1)]

DT1
#    col1 col_a col_b
# 1:    a     a  <NA>
# 2:  a_b     a     b
DT2
#    col1 col_a col_b
# 1:    a     a     a
# 2:  a_b     a     b

sessionInfo()
# R version 4.0.3 (2020-10-10)
# Platform: x86_64-apple-darwin13.4.0 (64-bit)
# Running under: macOS Big Sur 10.16
# 
# Matrix products: default
# LAPACK: /Users/xx/miniconda3/lib/libopenblasp-r0.3.12.dylib
# 
# locale:
# [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
# 
# attached base packages:
# [1] stats     graphics  grDevices utils     datasets  methods   base     
# 
# other attached packages:
# [1] data.table_1.13.6
# 
# loaded via a namespace (and not attached):
# [1] compiler_4.0.3 tools_4.0.3
2 Answers
data.table::tstrsplit(c('a', 'a_b'), '_')
[[1]]
[1] "a" "a"

[[2]]
[1] NA  "b"

data.table::tstrsplit("a", '_')
[[1]]
[1] "a"

When you don't group you get 1st result. When you group by id you get 2nd result and because you have 2 columns and 1 value, value gets repeated.

It seems that tsrtplit constructs a matrix of expected size and fill with NA when there is no value.

When you do the same thing in a group of size 1, trstplit returns only a which is recycled for the two columns to fill.

See this example:

library(data.table)

DT <- data.table(col1 = c('a', 'b'))
DT[, c('col2', 'col3') := 'a']
DT

## output
   col1 col2 col3
1:    a    a    a
2:    b    a    a
Related