Is it possible to parallelize the geocode function from the tidygeocoder package in R?

user_14448038 0
Viewed 72

I am trying to geocode a couple of addresses in R. I am working with the geocode function from the tidygeocoder package in R. To make it faster I'd love to parallelize this function, but I do not know how this might work.

I am working on Windows.

Here is an example:

id street state county
1 123 happiness TT South
2 234 parallel HH North 
# create cluster
cl <- parallel::makeCluster(8, type = "PSOCK")

# geocode the addresses
latlon <- parallel::parLapply(cl=cl, geocode, .tbl = address_df, method = 'osm', lat = latitude , long = longitude, address = NULL, street = street, county = county, state = state)

street should be the list (is a column of ) I'm trying to run the parLapply function on and geocode the function. The geocode function does not let me use a vector for addresses. I have use all the options of geocode, street, county and state to get their latitude and longitude.

Is it possible to parallelize this function?

parallel-processing r
1 Answers

Original Data Extended

I extended address_df by a real-world example:

address_df <- data.frame(id = c(1, 2, 3),
                         street = c("123 happiness", "234 parallel", "60 E Broadway"),
                         state =  c("TT", "HH", "MN"),
                         county = c("South", "North", "Hennepin"))

Packages and Purpose

For iteration I generally like to work with the purrr package (https://purrr.tidyverse.org/) and for parallelization I suggest its extension, the furrr package (https://furrr.futureverse.org/).

library(tidyverse) # includes purrr package
library(tidygeocoder)
# install.packages("furrr") # will also install `future`package
library(furrr)  # will also load `future`package

Data Preparation

Next to the geocode() function there is also the geo() function which only requires a comma-separated sting with a combined address as an argument, this is preferable for iteration, instead of .tbl and the arguments street, state, and county we will only need to pass one comma-separated address string.

address_df <- address_df %>% 
  mutate(address = paste(street, state, county, sep = ", "))

The iteration in furrr requires a list:

address_list <- address_df$address %>%
  as.list()

address_list now contains a list element for each comma-separated address sting.

Preparation for Parallelization

The plan() function from the future package (is installed and loaded with furrr package) defines how many cores you want to work on the parallelization, I usually chose availableCores() - 1

plan(strategy = "multisession", workers = availableCores() - 1)

Parallel Iteration

address_geodata <- future_map(.x = address_list, 
                              ~ geo(address = .x, method = 'osm', lat = latitude , long = longitude)) %>% 
  # puts the lists back together into a dataframe/tibble
  bind_rows()

Join Original data and geo data

address_df_geo <- address_df %>% 
                  left_join(address_geodata, by = "address")

Outcome Data Frame

address_df_geo

user_149023710
Related