How do you create/rename variables, change variable type, as well as filter rows on excel files that have been read into data frames in a loop in R?

Viewed 33

I wrote a script in R that reads in one excel file and then creates/renames variables, changes variable type, fills in values, as well as filters rows as necessary in order to clean up the file. My question is how do I apply that script to all excel files in a specific folder and then set them together once they've all been cleaned up.

The original script is:

library(xlsx)

install.packages("tidyverse")
library(tidyverse)

input_data_directory = '/Users/A/Documents/folder1'

#  UPDATE BEFORE RUNNING CODE
input_file  = '/wasde0822.xls'

# PROCESSING THE DATA
# reading in source file
wasde1 <- read.xlsx(paste0(input_data_directory, input_file),
                      sheetName = "Page 8",
                      colIndex = 1:8,
                      rowIndex = 9:33)

Date <- read.xlsx(paste0(input_data_directory, input_file),
                        sheetName = "Page 8",
                        header = F,
                        startRow = 1,
                        endRow = 1,
                        colIndex = 1)

# renaming fields
names(wasde1)[names(wasde1) == 'World'] <- "Product"
names(wasde1)[names(wasde1) == 'NA.'] <- "Time"
names(wasde1)[names(wasde1) == 'NA..1'] <- "Month"
names(wasde1)[names(wasde1) == 'Total.Supply'] <- "TotalSupply"
names(wasde1)[names(wasde1) == 'Trade.2.'] <- "Trade"
names(wasde1)[names(wasde1) == 'Total.Use.3.'] <- "TotalUse"
names(wasde1)[names(wasde1) == 'Ending.Stocks'] <- "EndingStocks"

# filtering out unnecessary rows
wasde1 = wasde1[-1,]
wasde1 <- wasde1[!is.na(wasde1$Time),]

# filling in values
wasde1 <- wasde1 %>% fill(Product, .direction = "down")

# creating new fields
wasde1$Geography <- "World"
wasde1$Date <- paste(Date)

# changing character fields to numeric fields
wasde1$Output <- as.numeric(wasde1$Output)
wasde1$TotalSupply <- as.numeric(wasde1$TotalSupply)
wasde1$Trade <- as.numeric(wasde1$Trade)
wasde1$TotalUse <- as.numeric(wasde1$TotalUse)
wasde1$EndingStocks <- as.numeric(wasde1$EndingStocks)

My attempts to put this in a loop to apply to more than one excel file were not successful. I'm able to read multiple excel files in a loop (below code, which works), but the rest has eluded me; any help would be appreciated. I'm sure this is possible; it's my R knowledge that is lacking.

library(xlsx)

install.packages("tidyverse")
library(tidyverse)

input_data_directory = '/Users/A/Documents/folder1'

filename_list <- list.files(input_data_directory,pattern="*.xls", full.names = FALSE)


for (i in 1:length(filename_list)) {

# reading in source file
  assign(filename_list[i], read.xlsx(paste0(input_data_directory,filename_list[i]),
                                 sheetName = "Page 8",
                                 colIndex = 1:8,
                                 rowIndex = 9:33))
  
  
  
  assign(paste("date",i,sep=""), read.xlsx(paste0(input_data_directory, filename_list[i]),
                                           sheetName = "Page 8",
                                           header = F,
                                           startRow = 1,
                                           endRow = 1,
                                           colIndex = 1))
}


Apologies in advance for not providing sample data. The best I'm able to do is provide a link to one sample file. This link will take you to a U.S. Department of Agriculture page. On that page, click on the excel report link circled below to get a sample file:

Link to U.S. D of A site

enter image description here

0 Answers
Related