Building data frame of flood metric calculations with a for loop in R

user_14902503 0
Viewed 58

I have a dataset called all.cols2 with water depths taken every 20 minutes for 94 locations over 3+ years. Here is a preview:

 # A tibble: 89,714 x 95
   date_time           Levee.slope      Levee.slope.1      Levee.slope.2    Levee.slope.3
   <dttm>                         <dbl>            <dbl>            <dbl>            <dbl>
 1 2015-12-01 15:05:33           -0.821           -0.539           -0.325          -0.0991
 2 2015-12-01 15:25:33           -0.830           -0.548           -0.334          -0.108 
 3 2015-12-01 15:45:33           -0.829           -0.547           -0.333          -0.107 
 4 2015-12-01 16:05:33           -0.833           -0.551           -0.337          -0.111 
 5 2015-12-01 16:25:33           -0.829           -0.547           -0.333          -0.107 
 6 2015-12-01 16:45:33           -0.834           -0.552           -0.338          -0.112 
 7 2015-12-01 17:05:33           -0.839           -0.557           -0.343          -0.117 
 8 2015-12-01 17:25:33           -0.835           -0.553           -0.339          -0.113 
 9 2015-12-01 17:45:33           -0.826           -0.544           -0.330          -0.104 
10 2015-12-01 18:05:33           -0.804           -0.522           -0.308          -0.0821
# ... with 89,704 more rows, and 90 more variables: Levee.slope.4 <dbl>,

I am calculating metrics for individual flooding events at each location.

Right now I have been using the for loop below to calculate these metrics one location at a time, exporting the results and copying and pasting them into an excel file, which takes a long time. Here is the code I've been using:

for (col in 1:length(list.sites)))
  #Label and subset by site  
  site <-  paste0("WaterLevel_",noquote(list.sites[[1]][i])) 
  mut_sub <- all.cols2 %>% select("Date",all_of(site))
  
  # creates binary for positive/negative water level values 
  mut_sub$VarA <- as.integer(mut_sub[,2] > 0) 
  
  # This code is used to label flood events with unique streak_id
  mut_sub <- mut_sub %>% mutate(lagged = lag(VarA))
  mut_sub<-  mut_sub%>% mutate(start = (VarA != lagged)) 
  mut_sub[1, "start"] <- FALSE 
  #filter to keep positive water depths (VarA == 1)
  mut_sub <- mut_sub %>% mutate(streak_id = cumsum(start)) %>%
    filter(VarA == 1)
 
  #calculate mean water depth
  ls <- aggregate(mut_sub[,2], by= list(mut_sub$streak_id), FUN = mean, na.rm = TRUE) 
  
  names(ls)[2] <- "avg_water_depth" 
  
  #calculate max water depth
   MAX <- aggregate(mut_sub[,2], by = list(mut_sub$streak_id), FUN = max, na.rm = TRUE)
   
   names(MAX)[2] <- "max_depth"
  
  #getting length (# of observations) of each event
  obs <- aggregate(mut_sub[,2], by = list(mut_sub$streak_id), FUN = length)
  
  names(obs)[2] <- "observations"
  
  #calculating number of days per event (duration)
  obs <- obs %>%
    mutate(duration_days = (((observations-1)*20)/60)/24)
  
  #Time interval: 
  time <- mut_sub %>% group_by(streak_id) %>% summarise(begin = min(date_time), end = max(date_time))
  time <- time %>% rename(Group.1 = streak_id)
  
  #combine data
  results1 <- inner_join(ls, MAX)
  results2 <- inner_join(results1, obs)
  final <- inner_join(results2, time)

 #way to label sites
  final$site = paste(site, final$Group.1, sep = "_")
}

###...repeat above for each survey point, export and add manually in excel

This gives an output that looks like this (this is from one site):

 Group.1 avg_water_depth   max_depth observations duration_days      begin        end                        site
      1     0.025245673 0.033995673            4    0.04166667 2016-02-09 2016-02-09  WaterLevel_Levee.slope.1_1
      3     0.045995673 0.071995673            8    0.09722222 2016-05-06 2016-05-06  WaterLevel_Levee.slope.1_3
      5     0.003995673 0.005995673            2    0.01388889 2016-05-06 2016-05-06  WaterLevel_Levee.slope.1_5
      7     0.039370673 0.061995673            8    0.09722222 2016-05-07 2016-05-07  WaterLevel_Levee.slope.1_7
      9     0.038785147 0.069995673           19    0.25000000 2016-05-27 2016-05-27  WaterLevel_Levee.slope.1_9
     11     0.063817102 0.110995673           28    0.37500000 2016-05-27 2016-05-28 WaterLevel_Levee.slope.1_11
     13     0.062817102 0.112995673           28    0.37500000 2016-05-28 2016-05-28 WaterLevel_Levee.slope.1_13
     15     0.042495673 0.067995673           18    0.23611111 2016-05-28 2016-05-28 WaterLevel_Levee.slope.1_15

...where each flood event at each location has an average water depth, maximum water depth, # of observations, duration in days of the flood event, and the date/time it started and ended.

Right now I have to specify i before I run the for loop, it doesn't automatically go through my sites.

My question is, is there a way have to for loop run through all locations at once and store it in a combined output similar to the above table? Also, is there a way to condense the code that I do have in my loop so I don't have to create so many dataframes?

dplyr subset data-manipulation r loops
1 Answers

Without some data it hard to showcase but here is a psuedo code using foreach and if you want to speed thing up you can use doParallel

data <- bind_rows(foreach(location = list_locations) %do% {
  # code handling data for one location
  # ...
  
  # process for each column of one location
  one_location_df <- bind_rows(foreach(i_col=(1:length(data))) %do% {
    # your code handling data
    
    # the final return should be a data_frame even if it is one row data frame
    return(one_result_df)
  })
  
  # some additiona code if has
  # ...
  return(one_location_df)
})

Note: if use doParallel, avoid wrap %dopar% around another %dopar% or it will cause memory leaked and nothing work

user_42096570
Related