Connecting OAuth2 resource server with authentication server

I'm trying to make a sample OAuth2 Spring authorization and resource server. My intention is to implement two separate applications - one representing authorization server ant the other representing resource server. Since I'm quite a beginner in Spring Security, I guess I need some guidance to complete my task.

I already managed to implement a simple authorization server using in-memory token store (app named "OAuth").

AuthServerOAuth2Config.java

@Configuration
@EnableAuthorizationServer
public class AuthServerOAuth2Config extends AuthorizationServerConfigurerAdapter {
    private static final String RESOURCE_ID = "myResource";

    @Autowired
    private UserApprovalHandler handler;

    @Autowired
    @Qualifier("authenticationManagerBean")
    private AuthenticationManager authManager;

    @Override
    public void configure(ClientDetailsServiceConfigurer clients) throws Exception {
         // @formatter:off
         clients.inMemory()
                 .withClient("test")
                    .authorizedGrantTypes("password", "authorization_code", "refresh_token", "implicit")
                    .authorities("ROLE_CLIENT", "ROLE_TRUSTED_CLIENT")
                    .scopes("read", "write", "trust")
                    .resourceIds(RESOURCE_ID)
                    .secret("test")
                    .accessTokenValiditySeconds(300).//invalid after 5 minutes.
                    refreshTokenValiditySeconds(600);//refresh after 10 minutes.
         // @formatter:on
    }

    @Override
    public void configure(AuthorizationServerEndpointsConfigurer endpoints) throws Exception {
        endpoints.tokenStore(tokenStore()).userApprovalHandler(handler).authenticationManager(authManager);
    }

    @Bean
    public TokenStore tokenStore() {
        return new InMemoryTokenStore();
    }

}

OAuth2SecurityConfig.java

@Configuration
@EnableWebSecurity
public class OAuth2SecurityConfig extends WebSecurityConfigurerAdapter {
    private static final Logger LOG = LoggerFactory.getLogger(OAuth2SecurityConfig.class);

    @Autowired
    private ClientDetailsService clientService;

    @Autowired
    private DataSource dataSource;

    @Autowired
    public void globalUserDetails(AuthenticationManagerBuilder auth) throws Exception {
        // @formatter:off
        auth.inMemoryAuthentication()
        .withUser("javabycode").password("123456").roles("USER")
        .and()
        .withUser("admin").password("admin123").roles("ADMIN");
        // @formatter:on
    }

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        // @formatter:off
         http
         .csrf().disable()
         .anonymous().disable()
            .authorizeRequests()
            .antMatchers("/oauth/token").permitAll();
        // @formatter:on
    }

    @Override
    @Bean
    public AuthenticationManager authenticationManagerBean() throws Exception {
        return super.authenticationManagerBean();
    }

    @Bean
    public TokenStore tokenStore() {
        return new JdbcTokenStore(dataSource);
    }

    @Bean
    @Autowired
    public TokenStoreUserApprovalHandler userApprovalHandler(TokenStore tokenStore) {
        TokenStoreUserApprovalHandler handler = new TokenStoreUserApprovalHandler();
        handler.setTokenStore(tokenStore);
        handler.setRequestFactory(new DefaultOAuth2RequestFactory(clientService));
        handler.setClientDetailsService(clientService);
        return handler;
    }

    @Bean
    @Autowired
    public ApprovalStore approvalStore(TokenStore tokenStore) throws Exception {
        TokenApprovalStore store = new TokenApprovalStore();
        store.setTokenStore(tokenStore);
        return store;
    }

}

Accessing http://localhost:9081/OAuth/oauth/token?grant_type=password&username=admin&password=admin123 returns token as expected, so I'm guessing that authorization server is configured ok.

Now there's a resource server part (app named "RestTest"). I've managed to find some examples using RemoteTokenServices to access token service that resides in another app. So here's my resource server so far.

OAuth2ResourceConfig.java

@Configuration
@EnableResourceServer
@EnableWebSecurity
public class OAuth2ResourceConfig extends ResourceServerConfigurerAdapter {
    private static final String RESOURCE_ID = "myResource";

    private TokenExtractor tokenExtractor = new BearerTokenExtractor();

    @Override
    public void configure(HttpSecurity http) throws Exception {
        // @formatter:off
        http.
        anonymous().disable()
        .requestMatchers().antMatchers("/v1/**")
        .and().authorizeRequests()
        .antMatchers("/v1/**").access("hasRole('ADMIN')")
        .and().exceptionHandling().accessDeniedHandler(new OAuth2AccessDeniedHandler());
        // @formatter:on
    }

    @Override
    public void configure(ResourceServerSecurityConfigurer resources) throws 
        Exception {
         resources.tokenServices(tokenService()).resourceId(RESOURCE_ID).stateless(true);
    }

    @Primary
    @Bean
    public RemoteTokenServices tokenService() {
        RemoteTokenServices tokenService = new RemoteTokenServices();
        tokenService.setCheckTokenEndpointUrl("http://localhost:9081/OAuth/oauth/check_token/");
        tokenService.setClientId("test");
        tokenService.setClientSecret("test");
        return tokenService;
    }
}

I'm trying to secure my REST API (http://localhost:9081/RestTest/v1/foobar) so I believe that configuration above is correct, right? Problem is that when I access v1/foobar endpoint (via Postman) it's accessible without any authentication. So I think I'm simply missing some part of configuration, but I can't figure it out how to connect to authorization server correctly. One more thing to mention - I'm not using Spring Boot!

I'd really appreciate some guidance to make my sample work. Thanks!

EDIT1: I've added resourceId to both authentication and resource server - no luck. Is resourceId even mandatory?