Keycloak REST API 401 when using custom client with service account

we want to access the Keycloak (Version 10) REST API from one of our backend services. To authenticate, we have setup the followings:

• new clients within our realm
• enabled service account for that client
• assigned all [1] roles of “realm-management” to the services account

Accessing the API, e.g. fetching a selected user always results in a 401 response.

Steps to make the requests are:

1. Retrieve access_token from https://my-keycloak.com/auth/realms/my-realm/protocol/openid-connect/token using grant_type=client_credentials + Client ID + Client Secret
2. Fetch user from https://my-keycloak.com/auth/realms/my-realm/users/some-user-id-4711 using the Authorization: Bearer $ACCESS_TOKEN with the Token from step 1.

My Question: Is it even possible to use a custom client or do we have to stick to login via admin-cli? How would we need to configure the custom client, to grant access to the REST API.

Thanks, Martin

[1] Simply setting all roles for the sake of testing, regardless that we only want to read data in the end.