httr::GET returns error for a specific website

user_13767899 0
Viewed 359

I am trying to web scrape the information from a database using the httr package in R. Every time I try to connect to it to get the HTML content I get the following error.

url <- "https://metalpdb.cerm.unifi.it"

query <- httr::GET(url)

Error in curl::curl_fetch_memory(url, handle = handle) : 
  schannel: next InitializeSecurityContext failed: SEC_E_ILLEGAL_MESSAGE (0x80090326) - This error usually occurs when a fatal SSL/TLS alert is received (e.g. handshake failed).

What could be the problem here? It seems to be related to this specific website. I am by no means an expert when it comes to this and have never tried to web scrape something before. I have never encountered an error like this when using httr::GET.

Help would be highly appreciated!

> sessionInfo()
R version 4.1.3 (2022-03-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19043)

Matrix products: default

locale:
[1] LC_COLLATE=English_Switzerland.1252  LC_CTYPE=English_Switzerland.1252   
[3] LC_MONETARY=English_Switzerland.1252 LC_NUMERIC=C                        
[5] LC_TIME=English_Switzerland.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] httr_1.4.2

loaded via a namespace (and not attached):
[1] compiler_4.1.3 R6_2.5.1       tools_4.1.3    curl_4.3.2
rvest httr web-scraping r
1 Answers

When I run your code it actually works:

library(httr)
url <- "https://metalpdb.cerm.unifi.it"

query <- httr::GET(url)
query

Output:

Response [https://metalpdb.cerm.unifi.it/]
  Date: 2022-04-04 11:58
  Status: 200
  Content-Type: text/html; charset=UTF-8
  Size: 22.5 kB


<!DOCTYPE html>
<html>
    <head>
        <title>Welcome to MetalPDB</title>
        <script src="/assets/javascripts/jquery-1.11.1.min.js" type="text/javascript"></script>
        <script src="/assets/javascripts/jquery-ui.min.js" type="text/javascript"></script>
        <script src="/assets/javascripts/collapse.js" type="text/javascript"></script>
        <script src="/assets/javascripts/transition.js" type="text/javascript"></script>
...

What you could do is restart R then install.packages(c("curl", "httr")) again.

Session info:

R version 4.1.0 (2021-05-18)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS 12.3.1

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] nl_NL.UTF-8/nl_NL.UTF-8/nl_NL.UTF-8/C/nl_NL.UTF-8/nl_NL.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] httr_1.4.2

loaded via a namespace (and not attached):
[1] compiler_4.1.0 R6_2.5.1       tools_4.1.0    curl_4.3.2
user_142827140
Related