Our project is using dataproc, and I have checked for service account that is indicated in master and worker compute engines that it has required permissions for my bucket storage. Nevertheless, in log viewer I have seen following:
(gsutil) Failed to create the default configuration. Ensure your have the correct permissions on: [/home/.config/gcloud/configurations]. Could not create directory [/home/.config/gcloud/configurations]: Permission denied. Please verify that you have permissions to write to the parent directory.
What can I do to fix this issue? I should download some file from this bucket and using gsutil to do that.
PS: I have tried using action.sh to find out logged in user which results in nothing.
PS2: We are using SparkSession for this operation.