TokenResponseException and userRateLimitExceeded error after upgrading Google Drive Api from v2 to v3

Question

We are using the Google Drive API, 6 months ago, we upgraded the Google Drive Api from v2 to v3, since then we received some reports from user saying writing to Google Sheet in our application was not working.

We also noticed invalid_grant (TokenResponseException 400 Bad Request) and userRateLimitExceeded (GoogleJsonResponseException 403 Forbidden) in our backend log system.

In Google Console page, there is only 403 error from drive.files.create, drive.files.get and drive.files.list in last 30 days.

I am not sure whether our problem is related to Drive Api version (v2, v3). I tried some fixes for this, but we still have this problem right now.

TokenResponseException

Since the Credential.java will write the log, we noticed some logs:

com.google.api.client.auth.oauth2.TokenResponseException: 400 Bad Request
{
  "error" : "invalid_grant"
}
    at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:105)
unable to refresh token

some error has the error_description:

com.google.api.client.auth.oauth2.TokenResponseException: 400 Bad Request
{
  "error" : "invalid_grant",
  "error_description" : "Token has been expired or revoked."
}
    at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:105)
unable to refresh token

Here is the code we create the credential:

val drive = new Drive.Builder(
  transport,
  jsonFactory,
  getCredentials(token.access_token, token.refresh_token)
).setApplicationName(SpreadsheetsHelper.CONSTANTS.SPREADSHEET_NAME).build()

private def getCredentials(accessToken: String, refreshToken: String) = {
  new GoogleCredential.Builder()
    .setClientSecrets(App.googleClientId, App.googleClientSecret)
    .setJsonFactory(jsonFactory)
    .setTransport(transport)
    .build()
    .setAccessToken(accessToken)
    .setRefreshToken(refreshToken)
}

We read many posts from Stackoverflow, but haven't fixed it.

We confirmed that some users with this Drive token issue didn't revoke the access from their Google Account page. And for every request of Drive Api, we create the credential like the code above.

However we didn't find any 400 error under Drive Api in Google Console admin page.

User Rate Limit Exceeded

For all 403 Forbidden, the reason in error log is 'User Rate Limit Exceeded':

com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
  "code" : 403,
  "errors" : [ {
    "domain" : "usageLimits",
    "message" : "User Rate Limit Exceeded",
    "reason" : "userRateLimitExceeded"
  } ],
  "message" : "User Rate Limit Exceeded"
}
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469) ~[com.google.api-client.google-api-client-1.22.0.jar:1.22.0]

Our quota of query is always:

Queries per day: 1,000,000,000
Queries per 100 seconds per user: 1,000 (the maximum value by default)
Queries per 100 seconds: 10,000

We tried to use quoteUser parameter according to the document, but seems it still not working:

val request = drive.files().list().setQ(query).setQuotaUser(quotaUser)
request.execute()

References:

https://developers.google.com/drive/v3/web/handle-errors#403_user_rate_limit_exceeded

https://developers.google.com/drive/v3/web/query-parameters

Downgrade Drive Api from v3 to v2

Yesterday, we downgraded Drive Api from v3 to v2:

 lazy val googleD = Seq(
   "com.google.apis" % "google-api-services-oauth2" % "v2-rev126-1.22.0",
-  "com.google.apis" % "google-api-services-drive" % "v3-rev71-1.22.0",
+  "com.google.apis" % "google-api-services-drive" % "v2-rev169-1.20.0",
   "com.google.apis" % "google-api-services-gmail" % "v1-rev64-1.22.0",
   "com.google.gdata" % "core" % "1.47.1"
 )

But this morning, I checked Google Console page, we still have 403 Forbidden from drives.files.insert. The insert() is only for v2, in Drive Api v3, this method is already changed to create().

Reference:

https://developers.google.com/drive/v3/web/migration

What we want to do next

How can we retrieve the Errors report in Google Console admin page more than 30 days before, so that we can confirm whether this is our legacy issue, or it just happened after we upgraded from Drive v2 to v3?
I noticed com.google.gdata" % "core" % "1.47.1" is an old Sheet Api, do we need upgrade Sheet Api to latest version (such as v4)?
I am asking the help from community support.

If you have existing apps based on the Google Sheets API v3, you can migrate to Google Sheets API v4. The v4 version is JSON-based, has an easier-to-use interface, and adds a substantial amount of functionality that is not possible in the v3 version.