Dataproc operation failure: INVALID_ARGUMENT: User not authorized to act as service account

This error is related to the lack of Service Account user role (roles/iam.serviceAccountUser) associate to the user/service account used to run the DataProc job.

In order to overcome this error, you need to go to the IAM Policy Console and give the Service Account User role, as described here, to the current user/service account you are using to run the job. As exemplified below:

1. Go to the IAM & Admin Console
2. Click on IAM
3. Select the member you are using to run your job
4. Click on the pen icon in the right side of the member's info
5. Add the Service Account user role

Pointing out some important topics, service accounts are used to make authorised API calls, through the service account itself or through delegated users within it. Moreover, about impersonation service accounts, an user with particular permissions can act as another service account with the necessary permission to execute a specific job.

Note: in step 3, you can also give to a particular user(email) the roles/iam.serviceAccountUser by clicking on +ADD (in top of the console). Then, writing the email and selecting the permission. Although, I must stress that this permission would be given at a project level. Thus, this user will be able to impersonate any of the existent Service Accounts.

I would like to add that the Service Account User role should be added to the Cloud Data Fusion Service Account that was automatically generated by Data Fusion when the API was enabled.

In my case the member is called something like:

service-############@gcp-sa-datafusion.iam.gserviceaccount.com

And it's name is: Cloud Data Fusion Service Account

In my case, the error was misleading. Even though the error stated that the issue was with missing privileges for 'XXXXXXXX-compute@developer.gserviceaccount.com' user, the actual problem was only fixed after I granted the role to service-############@gcp-sa-datafusion.iam.gserviceaccount.com user. I followed the steps that @Alexandre Moraes provided to grant the correct level of access to the Cloud Data Fusion Service Account.

Yes, the error message is pointing to wrong- service account( i.e. 5086XXXX8XXX-compute@developer.gserviceaccount.com'). Actually the service account that was running the process was different. It could be found from logs or authorizationInfo attribute of request in the logs(as shown in attached image):

CreateCluster
us-west1:cdap-getmember-xxxxxxx-2fff-11eb-98ba-6a77bxxxx9dc
service-5086XXXX8XXX@gcp-sa-datafusion.iam.gserviceaccount.com

You need to add role(any one of OWNER,EDITOR,SERVICE ACCOUNT) to this account (i.e. service-5086XXXX8XXX@gcp-sa-datafusion.iam.gserviceaccount.com).

I have run into the same issue. Above responses have been very helpful. I had to use couple of additional resources though to solve the problem:

1. Granting service account user permission
2. I had difficulty in finding the service account used by DataFusion for Dataproc cluster. I had to check the "Include Google-provided role grants" check box on the top right hand corner for it to show up.