GDrive export using Service Account creds fails with 404

Question

I have a script to export text from a GDrive file using an OAuth client, which works perfectly well -

import googleapiclient.discovery as google

from apiclient.http import MediaIoBaseDownload

from google_auth_oauthlib.flow import InstalledAppFlow

from google.auth.transport.requests import Request

import datetime, io, os, pickle

Scopes=" ".join(['https://www.googleapis.com/auth/drive.file',
                 'https://www.googleapis.com/auth/drive.metadata',
                 'https://www.googleapis.com/auth/drive.readonly'])

TokenFile="token.pickle"

def init_creds(clientfile,
               scopes,
               tokenfile=TokenFile):            
    token=None
    if os.path.exists(tokenfile):
        with open(tokenfile, 'rb') as f:
            token=pickle.load(f)            
    if (not token or
        not token.valid or
        token.expiry < datetime.datetime.utcnow()):    
        if (token and
            token.expired and
            token.refresh_token):
            token.refresh(Request())
        else:
            flow=InstalledAppFlow.from_client_secrets_file(clientfile, scopes)
            token=flow.run_local_server(port=0)
        with open(tokenfile, 'wb') as f:
            pickle.dump(token, f)
    return token

def export_text(id,
                clientfile,
                scopes=Scopes):
    creds=init_creds(clientfile=clientfile,
                     scopes=scopes)
    service=google.build('drive', 'v3', credentials=creds)
    request=service.files().export_media(fileId=id,
                                         mimeType='text/plain')
    buf=io.BytesIO()
    downloader, done = MediaIoBaseDownload(buf, request), False
    while done is False:
        status, done = downloader.next_chunk()
        destfilename="tmp/%s.txt" % id
    return buf.getvalue().decode("utf-8")

if __name__=='__main__':
    print (export_text(id="#{redacted}"
                       clientfile="/path/to/oath/client.json"))

But it's a pain to have to go through the OAuth flow every time, and since it's only me using the script I want to simplify things and use a Service Account instead, following on from this post -

Google Drive API Python Service Account Example

My new Service Account script, doing exactly the same thing, is as follows -

import googleapiclient.discovery as google

from oauth2client.service_account import ServiceAccountCredentials

from apiclient.http import MediaIoBaseDownload

import io

Scopes=" ".join(['https://www.googleapis.com/auth/drive.file',
                 'https://www.googleapis.com/auth/drive.metadata',
                 'https://www.googleapis.com/auth/drive.readonly'])

def export_text(id,
                clientfile,
                scopes=Scopes):
    creds=ServiceAccountCredentials.from_json_keyfile_name(clientfile,
                                                           scopes)
    service=google.build('drive', 'v3', credentials=creds)
    request=service.files().export_media(fileId=id,
                                         mimeType='text/plain')
    buf=io.BytesIO()
    downloader, done = MediaIoBaseDownload(buf, request), False
    while done is False:
        status, done = downloader.next_chunk()
        destfilename="tmp/%s.txt" % id
    return buf.getvalue().decode("utf-8")

if __name__=='__main__':
    print (export_text(id="#{redacted}",
                       clientfile="path/to/service/account.json"))

but when I run it for the same id, I get the following -

googleapiclient.errors.HttpError:

It feels like the Service Account script is passing the authentication step (ie Service Account creds are okay) but then failing when trying to fetch the file - weird as I can fetch it fine using the OAuth version :/

Any thoughts on what might be causing this 404 error in the Service Account version, given the OAuth client version clearly works for the same id?

user_11551468 · Answer

Answer:

You need to share your file with the service account.

More Information:

As you would with any file, you need to give a user explicit permissions to be able to see it. As a service account is a separate entitiy to you, this goes for them as well.

Using the file sharing settings (you can just do this in the Drive UI by right-clicking the file and hitting Share), give the email address of the service account the correct permission (read/write). The email address of the service account is in the form:

service-account-name@project-id.iam.gserviceaccount.com

user_1841839 · Answer

Before making your call do a File.list to see which files the service account has access to. Doing a file.get on a file that the service account doesn't have access to will result in a file not found error. Remember that the service account is not you, it has its own google drive account. Any files you want to access need to be uploaded to its account or shared with the service account.

If the file.list fails then it would suggest to me that there is something wrong with the authorization and you should ensure that the service account has access to client file maybe its that file it cant find.

Granting service account acccess

Create a directory on your personal google drive account. Take the service account email address, it can be found in the key file you downloaded it has a @ in it. Then share that directory on your drive account with the service account like you would share with any other user.

Adding files to that directory may or may not give the service account access to them automatically permissions is a pain you may need to also share the file with the service account.
Remember to have the service account grant your personal account permissions to the file when it uploads it its going to be the owner.

GDrive export using Service Account creds fails with 404

2 Answers

Answer:

More Information:

Granting service account acccess