Google Auth token invalid_token error in Spring boot sample app

First time Spring security user here. Trying to authenticate a user through OAuth2 "Implicit grant" flow in a Spring boot app using Google. Here is the WebSecurityConfig code:

@EnableWebSecurity
public class WebSecurityConfigForTokenAuth extends WebSecurityConfigurerAdapter {

@Value("${spring.security.oauth2.resourceserver.jwt.issuer-uri}")
private String issuer;

@Value("${spring.security.oauth2.resourceserver.jwt.jwk-set-uri}")
private String jwkSetUri;

@Value("${google.client-id}")
private String clientId;

@Value("${google.iss}")
private String iss;

@Override
protected void configure(HttpSecurity http) throws Exception {
    http
        .sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS).and()
        .authorizeRequests()
        .anyRequest().authenticated().and()
        .oauth2ResourceServer()
        .jwt().decoder(jwtDecoder());
}

@Bean
JwtDecoder jwtDecoder() {
    NimbusJwtDecoder jwtDecoder = (NimbusJwtDecoder)
            JwtDecoders.fromOidcIssuerLocation(issuer);        

    OAuth2TokenValidator<Jwt> audienceValidator = new AudienceValidator(clientId);
    OAuth2TokenValidator<Jwt> withIssuer = JwtValidators.createDefaultWithIssuer(iss);
    OAuth2TokenValidator<Jwt> withAudience = new DelegatingOAuth2TokenValidator<> 
    (withIssuer, audienceValidator);

    jwtDecoder.setJwtValidator(withAudience);

    return jwtDecoder;
}

}

Here is the application.yml:

spring:
  security:
    oauth2:
      client:
        registration:
         google:
          clientId: ****.apps.googleusercontent.com
          clientSecret: 5uZ-****-
          scope:
           - email
           - profile
      resourceserver:
        jwt:
          issuer-uri: https://accounts.google.com
          jwk-set-uri: https://accounts.google.com/.well-known/openid-configuration

google:
  client-id: ***.apps.googleusercontent.com
  clientSecret: 5uZ-***-
  iss: accounts.google.com

I am able to retrieve the token using SoapUI, like this: Getting token using SoapUI

and I am sending that token using Authorization: Bearer header in the request to my server at http://localhost:8080/list, I am getting the error saying invalid_token: Invalid_token error Any idea why spring security thinks this token is invalid?