curl command behaves differently on Mac and Windows

user_5481267 0
Viewed 1036

I have some R code that works on a Mac but someone else is trying to run it on Windows and it doesn't seem to work. The code I have been using is:

for(i in 1:length(SNPs.needproxies)){
       print(i)
       system(paste0("curl -k -X GET 'https://ldlink.nci.nih.gov/LDlinkRest/ldproxy?var=",SNPs.needproxies[i],"&pop=MXL&r2_d=r2&token=",token,"' > ", dir,"out.",SNPs.needproxies[i],".txt"))
}

If I try to run this on Windows I get the error

https not supported or disabled'.

We have narrowed done the problem in that it seems to be that single quotes should be replaced with double quotes in the address. However as this is within a paste0() function, it thinks this is something else so we thought escaping the double quotes with a backslash i.e., \"https://ldlink.nci.nih.gov/LDlinkRest/ldproxy?var=",SNPs.needproxies[i],"&pop=MXL&r2_d=r2&token=",token,"\"

This kind of works in that it gives the output we want but it is supposed to print this to a file called something like dir/out.file.txt but it does not do this part.

The SNPs.needproxies is:

SNPs.needproxies <- c("rs709692","rs9659182","rs13064990","rs11130017","rs9832922" ,"rs36120363","rs4727815","rs7994762","rs72772387")

I have a token for the website but cannot share that but it is stored in the object called token.

r libcurl
3 Answers

Nothing to do with paste0() itself but with the Windows version of curl, which requires the https address to be enclosed within double- instead of single-quotes. Check here for a complete break down of this issue.

user_66488280

1. system() with curl -o option

After noting that curl on Windows requires the https address to be enclosed within double- instead of single-quotes, I would avoid the remaining piping issue altogether and use the -o option in curl to specify a file to write results to if you insist on writing the command out:

system(paste0("curl -k -X GET \"https://ldlink.nci.nih.gov/LDlinkRest/ldproxy?var=", SNPs.needproxies[i], "&pop=MXL&r2_d=r2&token=", token, "\" -o out.", SNPs.needproxies[i], ".txt"))

2. curl_download()

Alternatively, R has a few curl-based packages that take care of all of these details such as curl:

library(curl)
url <- sprintf("https://ldlink.nci.nih.gov/LDlinkRest/ldproxy?var=%s&pop=MXL&r2_d=r2&token=%s", SNPs.needproxies[i], token)
curl_download(url, sprintf("out.%s.txt", SNPs.needproxies[i]))

3. file.download()

You might also just use file.download() in this case as Konrad suggested:

url <- sprintf("https://ldlink.nci.nih.gov/LDlinkRest/ldproxy?var=%s&pop=MXL&r2_d=r2&token=%s", SNPs.needproxies[i], token)
download.file(url, sprintf("out.%s.txt", SNPs.needproxies[i]))

4. GET()

You could also use GET() in from the httr library:

library(httr)
u <- "https://ldlink.nci.nih.gov/LDlinkRest/ldproxy"
q <- list(var = SNPs.needproxies[i],
          pop = "MXL",
          r2_d = "r2",
          token = token)
f <- sprintf("out.%s.txt", SNPs.needproxies[i])
GET(url = u, query = q, write_disk(f))

5. LDproxy()

Note, there appears to be an R package specifically for connecting to this API here. In your case the code would be:

library(LDlinkR)
LDproxy(snp = SNPs.needproxies[i],
        pop = "MXL", 
        r2d = "r2", 
        token = token, 
        file = sprintf("out.%s.txt", SNPs.needproxies[i]))
user_93932130

The standard curl call in Windows is a wrapped version of PowerShell's Invoke-RestMethod, which is why doing system calls will result in different results depending on whether you use Linux or Windows. I would recommend using the httr package in R, though it will require translating the calls.

user_16522190
Related