How to run an R code within a given duration and then stop it?

user_12198638 0
Viewed 65

Currently I'm working on a web crawler project with Rselenium, which needs to open about 100,000 webpages in turn and do the info collection : 

url <- paste0("www.111", r0[ii],".com")
remDr$open()
remDr$navigate(url)

Seems that the most time-consuming portion of this process is opening a new webpage, especially loading ads, external links, etc. So how could I run this kind of R code within a specific duration (e.g., 2 seconds) and then stop it, and run the following info collection process? Thanks.

rselenium r
2 Answers

Ok, I withTimeout from R.utils seems to do what you want (interrupting a function after a delay).

library(R.utils)
A=2
foo <- function() {
while(A>1){print(A)}
}

#foo()#Ridiculous infinite function don't run it

withTimeout(foo(),timeout=0.5)
user_120830930

Got it! The R.utils package does work. Seems that the function withTimeout could be used along with the try function in order to continue running any following functions. e.g:

library(RSelenium)  
library(rvest)
library(R.utils)

remDr <- remoteDriver(remoteServerAddr = "127.1.1.1" 
                      , port = 4444
                      , browserName = "firefox") # connect to Server
remDr$open()
try(withTimeout(remDr$navigate("https://aaaa.org"), timeout=0.5)) # stop navigation after 0.5 sec
a1 <- read_html(remDr$getPageSource()[[1]][1])
a2 <- html_nodes(a1, "pre") %>% html_text()

Thanks!

user_121986380
Related