Download all the files (.zip and .txt) from a webpage using R

Here try this. It works because the file url has a repeatable pattern. It's a tad clunky getting the file names out of the webpage but it does seem to work.

Many text files may be missing an End of Line marker (this is common) and may throw an error. However, it is probably not an important error. If that happens, open the txt file that downloaded to ensure that it downloaded correctly. No doubt there is a way to automate that step but I'm out of time for this one, Dude (or Dudette or whatever you prefer).

#get homepage for locations
page <- "https://pubs.usgs.gov/sir/2007/5107/downloads/"
a <- readLines(page)

#find lines of interest
loc.txt <- grep(".txt", a)
loc.zip <- grep(".zip", a)

#A convenience function that uses
#line from original page
#marker of file type to locate name 
#and page (url original page)
#------------------------------------
convfn <- function(line, marker, page){
  i <- unlist(gregexpr(pattern ='href="', line)) + 6
  i2<- unlist(gregexpr(pattern =,marker,  line)) + 3
  #target file
  .destfile <- substring(line, i[1], i2[1])
  #target url
  .url      <- paste(page, .destfile, sep = "/")
  #print targets
  cat(.url, '\n', .destfile, '\n')
  #the workhorse function
  download.file(url=.url, destfile=.destfile)
  }
#--------------------------------------------

#they will save in your working directory
#use setwd() to change if needed
print(getwd())
  
#get the .txt files and download them
sapply(a[loc.txt], 
       FUN = convfn, 
       marker = '.txt"', #this is key part, locates text file name
       page = page)

#get the .zip files and download them
sapply(a[loc.zip], 
       FUN = convfn, 
       marker = '.zip"', #this is key part, locates zip file name
       page = page)