rbind is overwriting previous data in R

user_10961338 0
Viewed 163

So, I have had two questions here and I combined the solutions, but it seems I am not implementing them well. (Q1 Q2)

So, I am reading files using read_delim() and trying to filter out some data before saving them into one data type. 

data <- data.frame()
for (file in files){

  name = strsplit(file, split = "\\.")[[1]][1]

  tmp <- data %>% bind_rows(read_delim(file = file, delim = ";", col_types = cols(
    a = col_double(),
    b = col_double(),
    c = col_character()
    )) %>% mutate(filename = name)
    data_tmp <- tmp %>%
      filter(!str_detect(c, 'a'))
    data <- rbind(data, tmp_data)
}

Apparently, the problem I had in the first question has reappeared. Looks like that mutate(filename = name) always gets the value from the last element that was read.

r
1 Answers

We could make it more compact if we use map. Loop over the files, read the data with read_delim, create the 'filename' column with substring of the filename, filter the rows and convert it to a single data.frame with _dfr suffix

library(purrr)
library(dplyr)
out <- map_dfr(files, ~ {
          file <- .x
         read_delim(file, delim = ";", col_types = cols(
            a = col_double(),
            b = col_double(),
            c = col_character()
            )) %>%
          mutate(filename = str_remove(file, "\\..*")) %>%
          filter(!str_detect(c, 'a'))
        })

In the OP's for loop, the data %>% bind_rows is not needed because we are already rbinding the 'tmp_data` in each iteration

data <- data.frame()
for (file in files){

  name = strsplit(file, split = "\\.")[[1]][1]

  tmp <- read_delim(file = file, delim = ";", col_types = cols(
    a = col_double(),
    b = col_double(),
    c = col_character()
    )) %>% mutate(filename = name)
    tmp_data <- tmp %>%
      filter(!str_detect(c, 'a'))
    data <- rbind(data, tmp_data)
}
user_37322710
Related