How to append a column to data.frames in a list where the column shall contain computationally read out structural information of those data.frames?

user_11597479 0
Viewed 51

I have some data that I read in and modify in R. For a minimal, reproducible example (reprex) I want to give the data as an "in R" representation to also communicate the data structure:

The code to read in the data:

paths <- sprintf("filenames%02d.out", 1:26)
interim <- lapply(paths, read.table, header=FALSE, sep="\t", dec=".", na.strings="NA")
new_col_name <- c("Pos", "LRTD")
out <- lapply(interim, setNames, nm = new_col_name)

Now, lapply(out, head) allows us to see R's internal representation of the data:

[[1]]
     Pos LRTD
1      0    0
2  70557    0
3 104076    0
4 163349    0
5 258229    0
6 356613    0

[[2]]
     Pos LRTD
1      0    0
2 171603    0
3 268756    0
4 456513    0
5 594904    0
6 663581    0

[[3]]
     Pos  LRTD
1      0 0.000
2 171960 0.370
3 217096 0.358
4 254484 0.338
5 320866 0.366
6 432642 0.382

{...}

[[26]]
     Pos LRTD
1      0    0
2 185161    0
3 234971    0
4 273218    0
5 319689    0
6 379800    0

So it is a list of data.frames with 26 elements. Here, I want to call the numbers that we can see above in square brackets, so the numbers [[1]], [[2]], [[3]] and so forth till [[26]], as "element descriptors".

Now what I would like to do is append a third column to all the data.frames in the list where the column contains computationally read out structural information of the data.frames. In detail, I would like to add the element descriptors of the given data.frames to their respective data.frame. That in mind, the result should look like this:

[[1]]
     Pos LRTD   Chr
1      0    0   1
2  70557    0   1

[[2]]
     Pos LRTD   Chr
1      0    0   2
2 171603    0   2

[[3]]
     Pos  LRTD   Chr
1      0 0.000   3
2 171960 0.370   3

{...}

[[26]]
     Pos LRTD   Chr
1      0    0   26
2 185161    0   26

Since I am well aware of this question, my current solution is pseudocode:

lapply(out, function(x) { x$Chr <- rep("element descriptor","lenght of list");return(x)})

I know that I can get the length of the respective data.frame with rapply(out, length), but so far I don't get rapply to work within my lapply command from above.

Also, how to reference the element descriptor in code?

lapply dataframe append r
3 Answers

Map works well for this.

Map(function(x, ind) transform(x, Chr = ind), out, seq_along(out))
# [[1]]
#      Pos LRTD Chr
# 1      0    0   1
# 2  70557    0   1
# 3 104076    0   1
# 4 163349    0   1
# 5 258229    0   1
# 6 356613    0   1
# [[2]]
#      Pos LRTD Chr
# 1      0    0   2
# 2 171603    0   2
# 3 268756    0   2
# 4 456513    0   2
# 5 594904    0   2
# 6 663581    0   2
# [[3]]
#      Pos  LRTD Chr
# 1      0 0.000   3
# 2 171960 0.370   3
# 3 217096 0.358   3
# 4 254484 0.338   3
# 5 320866 0.366   3
# 6 432642 0.382   3
# [[4]]
#      Pos LRTD Chr
# 1      0    0   4
# 2 185161    0   4
# 3 234971    0   4
# 4 273218    0   4
# 5 319689    0   4
# 6 379800    0   4

If your "element descriptors" are really names, then replace that with

Map(function(x, ind) transform(x, Chr = ind), out, names(out))

and it will do effectively the same thing.

If you're comfortable with lapply and want to know how this compares with that, then the equivalent lapply for that Map would be:

lapply(names(out), function(nm) transform(out[[nm]], Chr = nm))

You can even code-golf it a bit, with

Map(transform, out, Chr = seq_along(out))
Map(transform, out, Chr = names(out))

(both identical output to above). This happens to work because we can use named arguments in Map that are passed-through to the f= (function) argument, transform in this case.

Data:

out <- list(structure(list(Pos = c(0L, 70557L, 104076L, 163349L, 258229L, 356613L), LRTD = c(0L, 0L, 0L, 0L, 0L, 0L)), class = "data.frame", row.names = c("1", "2", "3", "4", "5", "6")), structure(list(Pos = c(0L, 171603L, 268756L, 456513L, 594904L, 663581L), LRTD = c(0L, 0L, 0L, 0L, 0L, 0L)), class = "data.frame", row.names = c("1", "2", "3", "4", "5", "6")), structure(list(Pos = c(0L, 171960L, 217096L, 254484L, 320866L, 432642L), LRTD = c(0, 0.37, 0.358, 0.338, 0.366, 0.382)), class = "data.frame", row.names = c("1", "2", "3", "4", "5", "6")), structure(list(Pos = c(0L, 185161L, 234971L, 273218L, 319689L, 379800L), LRTD = c(0L, 0L, 0L, 0L, 0L, 0L)), class = "data.frame", row.names = c("1", "2", "3", "4", "5", "6")))
user_33582720

One option involving purrr could be:

imap(out, ~ update_list(.x, Chr = .y))

[[1]]
     Pos LRTD Chr
1      0    0   1
2  70557    0   1
3 104076    0   1
4 163349    0   1
5 258229    0   1
6 356613    0   1

[[2]]
     Pos LRTD Chr
1      0    0   2
2 171603    0   2
3 268756    0   2
4 456513    0   2
5 594904    0   2
6 663581    0   2

[[3]]
     Pos  LRTD Chr
1      0 0.000   3
2 171960 0.370   3
3 217096 0.358   3
4 254484 0.338   3
5 320866 0.366   3
6 432642 0.382   3

[[4]]
     Pos LRTD Chr
1      0    0   4
2 185161    0   4
3 234971    0   4
4 273218    0   4
5 319689    0   4
6 379800    0   4
user_59645570

Apparently, all files have a similar structure, i.e., the order and types of columns are identical (the names may differ). Therefore, I hope you don't mind if I suggest a completely different approach which doesn't answer your actual question but may help to solve what I believe is the underlying problem.

In situations like this, my preferred approach is to combine all files in one large dataset where the source is indicated for each row in an id column (OP's element descriptor Chr).

So, here is what I would do using my preferred toolset

library(data.table)
library(magrittr) 
new_col_name <- c("Pos", "LRTD")
paths <- sprintf("filenames%02d.out", 1:26)
out <- lapply(paths, read.table, header=FALSE, sep="\t", dec=".", na.strings="NA") %>%
  rbindlist(use.names = FALSE, idcol = "Chr") %>%
  setnames(old = 2:3, new = new_col_name)

    Chr    Pos  LRTD
 1:   1      0 0.000
 2:   1  70557 0.000
 3:   1 104076 0.000
 4:   1 163349 0.000
 5:   1 258229 0.000
 6:   1 356613 0.000
 7:   2      0 0.000
 8:   2 171603 0.000
 9:   2 268756 0.000
10:   2 456513 0.000
11:   2 594904 0.000
12:   2 663581 0.000
13:   3      0 0.000
14:   3 171960 0.370
15:   3 217096 0.358
16:   3 254484 0.338
17:   3 320866 0.366
18:   3 432642 0.382
19:   4      0 0.000
20:   4 185161 0.000
21:   4 234971 0.000
22:   4 273218 0.000
23:   4 319689 0.000
24:   4 379800 0.000
    Chr    Pos  LRTD

So, with all data in one large dataset, we can operate on whole columns without the need to call lapply() to deal with every chunk but we can group by or subset by Chr if necessary. This approach has simplified my workflows in many cases.

user_38170040
Related