I am trying to use the arrow R package to read a parquet file from s3. The documentation only describes how to specifying AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY when authenticating for access to private s3 bucket.
However, I have to generate keys using okta which additionally provides AWS_SESSION_TOKEN & AWS_SECURITY_TOKEN values as part of my profile.
How can arrow in R be instructed to use the AWS_SESSION_TOKEN value and AWS_SECURITY_TOKEN value as part of authentication? Or, how can arrow in R be instructed to use a profile instead?
I have tried setting environment variables:
Sys.setenv(AWS_ACCESS_KEY_ID="...", AWS_SECRET_ACCESS_KEY="...", AWS_SESSION_TOKEN="...", AWS_SECURITY_TOKEN="...")
library("arrow")
read_parquet("s3://bucket/file.extension?region=us-east-1")
But this results in
AWS Error [code 15]: No response body.
if I try to set the environment variable AWS_DEFAULT_PROFILE then the same error occurs.
Thank you in advance for your consideration and response.