invalid_grant error trying to refresh access token for googleapis

We use the googleapis for our web app and store and manage the access token and refresh token to our database. When we refresh the access token of users using refresh token, rarely we had the GaxiosError: invalid_grant.

Now, we use the google api nodejs client[ https://github.com/googleapis/google-api-nodejs-client ]. We store the access token and the refresh token to our database for each user and they are updated by bellow logic per 6 hours.

import { google } from 'googleapis';

// create oauth2client for google apis
const oAuth2Client = new google.auth.OAuth2(client_secret, client_id, redirect_uri);

// set current access token and refresh token (==current_tokens)
oAuth2Client.setCredentials(current_tokens);

// refresh access token and refresh token
// new_token contains access_token and refresh_token
const new_token = await oAuth2Client.refreshAccessToken();

// store the new access token and new refresh token to database
...

Does anyone know what may be causing GaxiosError: invalid_grant? I'm getting the feeling that it may be due to the refresh token being updated every 6 hours.

Additional Info

the setting of generating auth url

const authUrl = oAuth2Client.generateAuthUrl({
        access_type: 'offline',
        scope: GOOGLE_APIS_SCOPE, // GOOGLE_APIS_SCOPE contains scopes
        prompt: 'consent',
      });