Authorization_RequestDenied getting Group Members

I am using the GraphServiceClient in my code to access multiple endpoints for data.

I have a GraphServiceClient (with delegated permissions) with the following scope: Group.Read.All.

I also have a GraphServiceClient (with application permissions) with the following scopes: User.Read.All, Mail.Read.

I first get a Group Id with the first client, no problems here. Then I use the Group id in the following code:

var users = await client
    .Groups[id]
    .Members
    .Request()
    .GetAsync();

Now the following scope is needed Application - User.Read.All which I have on my 2nd client but when I use this client to get the members of this group it gives the following message:

Code: Authorization_RequestDenied 
Message: Insufficient privileges to complete the operation.

Yes, the permissions are granted by an admin in AD and I use the "graphresource"/.default scope (works with getting just list users).

Do I also need Directory.Read.All (can't test that right now) or am I missing something else?