I am trying to read a csv file using below code.
df = s3read_using(FUN= read.csv, bucket="my-bucket-name",object="object.csv")
While doing this I am getting the below error
Error in curl::curl_fetch_disk(url, x$path, handle = handle): "URL using bad/illegal format or missing URL"
I can reproduce the error, but I cannot explain it:
s3read_using(FUN = readRDS, object = "pfd.RDS", bucket = "collidr-api")
# Error in parse_aws_s3_response(r, Sig, verbose = verbose) :
# Moved Permanently (HTTP 301)
s3read_using(FUN = readRDS, object = "pfd.RDS", bucket = "collidr-api", region = "ap-southeast-2")
# Error in parse_aws_s3_response(r, Sig, verbose = verbose) :
# Moved Permanently (HTTP 301)
An alternative way to read in your data is like so (run the code below and it should work - you'll just have to replace bucket/file/region with your own values, and use read.csv() instead of readRDS().
bucket <- "collidr-api"
file <- "pfd.RDS"
region <- "ap-southeast-2"
s3location <- paste0("https://", bucket, ".s3-", region, ".amazonaws.com/", file)
readRDS(url(s3location))
# package_names function_names
# 1 A3 a
# 2 A3 a3
# 3 A3 A3-package
# 4 A3 a3.base
# 5 A3 a3.gen.default
# Data reads in successfully!
I noticed I got the same error message when I attempted to read a file that I didn't have permission to read. So that's another possibility. A very quick way of testing is to see if you can access the file directly in the browser, using the file's url (i.e. copy/paste the s3location variable above into your browser's address bar - if it downloads the file, there's no reason R should be able to do the same, but if it says access denied, then R will not be able to read that file without credentials for the bucket).
My input bucket name was like abc_res. Due to the naming convention I was getting the above error. I changed the bucket name as abc-res and error got resolved.