Loop for applying code to different files

user_10559646 0
Viewed 47

I hope you are all doing fine. Beginner here.

I have this code to get the number of rows and the mean of a column of an excel file.

data <- read.table("filename.xls",sep = "\t", header = T)

mean <- mean(data$Perim.)

ncells <- nrow(data)

df <- data.frame (mean,ncells)

write.table(df, file = "data.csv")`

I would like to find a way of running that code for all excel files in a folder, so I created a list of the excel files I would like the code applied to:

file.list <- list.files(pattern='*.xls')

But I can't even get past this part of the loop:

for(i in 1:length(file.list)){
  data[i] = read.table("file.list[[i]]",sep = "\t", header = T)
}

I get the following error:

Error in file(file, "rt") : cannot open the connection
In addition: Warning message:
In file(file, "rt") :
  cannot open file 'file.list[i]': No such file or directory

Could someone please explain this error and help build the rest of the loop? Very much appreciated.

r loops
2 Answers

Your file.list is a vector, not a list. So you should refer to its elements with single brackets: file.list[i] .

Then it is helpful to create a list item to store your results in. Note the use of double brackets on the list item mylist when you store a new list element.

mylist <- list()

for(i in 1:length(file.list)){
  mydata <- read.table("file.list[i]",sep = "\t", header = T)
  mean <- mean(mydata$Perim.)
  ncells <- nrow(mydata)
  mydata2 <- data.frame(Col1Label = mean, Col2Label = ncells)
  mylist[[i]] <- mydata2
} 

names(mylist) <- file.list

Then to access a given result, you can type:

mylist[[listnumber]]

or

mylist[["nameofelment"]]

You can use rbindlist() from the data.table package to collapse into a data.table if you so desire. This is highly efficient, even more so if you specify the size of the list first by knowing how many files you are going to parse.

You could also do the following where you created a dataframe first and use the loop to bind a single row to the dataframe each iteration. This is far from optimal with regards to performance, but if you only have 100's or 1000's of files to read through, it will be fine.

mydataframe <- data.frame()

for(i in 1:length(file.list)){
  mydata <- read.table("file.list[i]",sep = "\t", header = T)
  mean <- mean(mydata$Perim.)
  ncells <- nrow(mydata)
  mydataset <- file.list[i]
  mydata2 <- data.frame(Col1Label = mean, Col2Label = ncells, Dataset = mydataset)
  mydataframe <- rbind(mydataframe,mydata2)
}
user_40018970

Learn about apply or map. No need for a loop here:

x <- lapply(list.files(pattern='*.txt'), function(x) read.table(x ,sep = "\t", header = T))

x <- purrr::map(list.files(pattern='*.txt'), ~ read.table(.x, sep = "\t", header = T))

purrr::map(seq_along(x), ~ mean(data$perim[[.x]]))
user_124814530
Related