Writing a Custom Authorizer would be one of the most common best practices. This is also no bottlenet, because this Custom Authorizer would be itself a Lambda which dynamically scale up.
Iterate the development of your Custom Authorizer:
1. Lambda + Cognito
If you really prefer Cognito, than connect your Lambda to Cognito and request the user information from the event of the Lambda. Check if the user exists in Cognito and probably check, if the user is a member of a specific group.
This could be done using the SDK, e.g. boto3 for Python.
To reduce the API request to Cognito, use the Caching within API Gateway. Set this to e.g. 300 seconds. So only each five minutes the Cognito API will be requested.
2. Optimize Performance
If iteration 1 let's you conclude to this process will slow you down, than use the smallest redis instance and store the information you need to redis. Set the expire time to 30 minutes.
Change the behaviour of the Custom Authorizer to check first, if there is an entry in redis available. If yes, use this information if the values are not expired. If not, call the Cognito API and store the result also in redis.
Now you have system which will only call all 5 minutes to redis (because of Caching in API Gateway) and all 30 minutes the Cognito API, because of the expiration time of the objects within redis
3. Optimize stability
If all is working, you can work in the stability. E.g. check if redis is available (redis ping)...if not, do not throw errors, just proceed like you do not have redis.
If Cognito won't be reachable, extend the expired object to be sure, users are able to login while you solving the Cognito issues.
4. optional: enrich your data
you can enrich the data, like further checks by adding a DynamoDB and request those data. Store those information also in redis, will reduce the duration of the execution time of the Custom Authorizer Lambda itself.
Summarize
All in all, using Cache in API Gateway, Redis to reduce requests to a user backend (like Cognito) and reduce single point of failures in your dependencies (like check with redis ping) you can have a Custom Authorier which will check within less then 100ms, if a user is able to login/call an API or not.
Out of the box, you only have API Key for "userless" authorization. The limit of 10.000 is increaseable, but you also have to implement a service to attach a Key to a user... this would be possible by using triggers in Cognito User Pool (if user will be created, create new API Key and send it to the user using SES)...but you have to write further triggers/functions for use cases like remove a user, etc...
