I have four vectors in a list, where each vector contains 92 paths elements to SPSS data files that I wish to combine together and convert into an RDS.
Each vector of SPSS data files should first be read and combined into dataframe by row, and then the four resulting data frames should be joined by columns. I want to do something like this:
library(dplyr)
# 'pathlistX' are vectors, each containing 92 path elements
paths <- list(pathlist1, pathlist2, pathlist3, pathlist4)
my_df <- lapply(paths, function(x) {
lapply(x, haven::read_spss) %>%
reduce(bind_rows)
}
) %>%
reduce(full_join) %>%
saveRDS(., "/path/to/folder/my_data.RDS")
However, this is rather problematic as I run out of memory (32 GB RAM). I am uncertain why the process demand so much memory, as the total sixe of all SPSS files are ~2.5GB. I am therefore looking for a more memory efficient way to accomplish this task.