Creating a loop to download and write met data to csv

user_12367786 0
Viewed 75

I'm quite a novice at using R but I'm trying to self-teach and learn as I go along. I'm trying to create a loop to download and save multiple met data files individually as csv files using the worldmet package.

I have two variables, the met site code and the years of interest. I have included code to create a list of the years in question:

Startyear <- "2018"
Endyear <- "2020"

Yearlist <- seq(as.numeric(Startyear), as.numeric(Endyear))

and I have a .csv file with all the site codes listed which are required, and have read this into R. See below a simplified version of the dataframe, however in total there are 204 rows. This dataframe is called 'siteinfo'.

code          station                ctry
037760-99999  GATWICK                UK
037690-99999  CHARLWOOD              UK
038760-99999  SHOREHAM               UK
038820-99999  HERSTMONCEUX WEST END  UK
037810-99999  BIGGIN HILL            UK

An example of the code to import one years worth of metdata for one site is as follows

importNOAA(code="037760-99999",year=2019,hourly=TRUE,precip=FALSE,PWC=FALSE,parallel=FALSE,quiet=FALSE)

I understand that I likely need a nested loop to change both variables, but I am unsure if I am going about this correctly. I also understand that I need to have quotation marks around the code value for it to be read correctly, however I was wondering if there's a quick way to include this as part of the code rather than editing all 204 values in the csv?

Would I also need a separate loop following downloading the files, or can this be included into one piece of code?

The current code I have, and I am sure there is a lot wrong with this so I appreciate any guidance, is as follows

for(i in 1:siteinfo$code) {
  for(j in 1:Yearlist){
  importNOAA(code=i,year=j,hourly = TRUE, precip= FALSE, PWC= FALSE, parallel = TRUE, quiet = FALSE)
  }}

This currently isn't working, so if you could help me piece this together, and if possible provide any explanation of where I have gone wrong or how I can improve my coding, I would be extremely grateful!

csv r for-loop loops
2 Answers

You can avoid loops altogether (better for large data sets and files) with some functions in dplyr and purrr. I get an error for invalid parameters when I try to run your importNOAA code, so I am using a simpler call to that function.

met_data <- siteinfo %>%
    full_join(data.frame(year = Yearlist), by = character(0)) %>%
    group_by(code, year) %>%
    mutate(dat = list(data.frame(code, year))) %>%
    mutate(met = purrr::map(dat, function(df) {
      importNOAA(code = df$code, year = df$year, hourly=TRUE, quiet=FALSE)
    }) ) %>% 
    select(-dat)

This code returns a tbl.df where the last column is a list of data.frames, each containing the data for a year-code combination. You can use met_data %>% summarize(met) to expand the data into one big data.frame to save to a csv, or if you want to write them all to indidividual csvs, use lapply:

lapply(1:nrow(met_data), function(x) {
             write.csv(met_data$met[x], 
                       file = paste(met_data$station[x], "_", met_data$year[x], ".csv", sep = ""))})
user_129294470

you can't use for loop like for(i in 1:siteinfo$code){}...

just short example


for(i in 1:mtcars$mpg){
  print(i)
}

output:

numerical expression has 32 elements: only the first used[1] 1
[1] 2
[1] 3
[1] 4
[1] 5
[1] 6
[1] 7
[1] 8
[1] 9
[1] 10
[1] 11
[1] 12
[1] 13
[1] 14
[1] 15
[1] 16
[1] 17
[1] 18
[1] 19
[1] 20
[1] 21

So use just index like this

for(i in 1:nrow(siteinfo$code){
 for(j in 1:nrow(Yearlist){
   importNOAA(code=siteinfo$code[i],year=Yearlist[j],hourly = TRUE, precip= FALSE, PWC= FALSE, parallel = TRUE, quiet = FALSE)
  }}

maybe that's works

user_125260990
Related