How to validate Google access token locally using google oAuth libraries

I'm trying to use Google's APIs to modify data on my users' Google account through the use of an id_token for authentication and an access_token to actually use Google's APIs. I know I'm able to verify the authenticity of an id token like such:

import { OAuth2Client } from "google-auth-library";

const client = new OAuth2Client(GOOGLE_CLIENT_ID);
const ticket = await client.verifyIdToken({
    token: idToken,
    audience: GOOGLE_CLIENT_ID,
});

This verification happens locally on my device without needing to contact Google's servers each time a token needs to be verified.

I tried to figure out how to do the same for the access_token. The top answer on How can I verify a Google authentication API access token? post suggests that I should call an endpoint https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=accessToken to do the verification but that defeats my purpose of trying to do it locally.

What Google OAuth library/method can I use to verify an access token locally. Is it even possible?

Just to reiterate, I'm talking about the access_token, not the id_token.