r curl curl_download doesn't append

user_5943539 0
Viewed 124

I'm trying to take a set of dois and have the doc.org website return the information in .bib format. The code below is supposed to do that and crucially append each new result to a .bib file. mode = a is what I understand will do the appending but it doesn't. The last line of code prints out the contents of oufFile and it contains only the last .bib results.

What needs to be changed to make this work.

library(curl)
outFile <- tempfile(fileext = ".bib")
url1 <- "https://doi.org/10.1016/j.tvjl.2017.12.021"
url2 <- "https://doi.org/10.1016/j.yqres.2013.10.005"
  h <- new_handle()
    handle_setheaders(h, "accept" = "application/x-bibtex")

curl_download(url1, destfile = outFile, handle = h, mode = "a")
curl_download(url2, destfile = outFile, handle = h, mode = "a")
read_delim(outFile, delim = "\n")
r curl
1 Answers

It's not working for me as well with curl_download(). Alternatively you could download with curl() and and use write() with append = TRUE.

Here is a solution for that, which easily can be used for as many urls as you are looking to download the bibtex from. You can execute this after your line 7.

library(dplyr)
library(purrr)
urls <- list(url1, url2)
walk(urls, ~ {
  curl(., handle = h) %>%
    readLines(warn = FALSE) %>%
    write(file = outFile, append = TRUE)
})

library(readr)
read_delim(outFile, delim = "\n")
user_48211420
Related