I have been trying to read some data from an AWS S3 bucket to Databricks. The S3 bucket is development/team/user. I am using the Scala statement:
val test = spark.read.format("team").load("/mnt/development/team/user/data.txt")
and I get the following:
com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: User [user] is not authorized to perform sts:AssumeRole on resource: [resource]
How do I resolve this error? Is there a problem with the Databricks cluster I am running? Is there a way to allow my Databricks role to perform sts:AssumeRole in AWS? I am not sure if or how I can do either.