I'm looking to emulate the Python boto3 module as a means of providing a lightweight emulation of AWS services (SQS, S3, Parameter Store etc.) for boto3 clients transparently.
Context:
I want to do this mainly because, my team provides a Python module which wraps these AWS functionalities which is consumed by many of these services. So, as a means of enabling local testing before their services can be run on AWS, this emulation layer would be a means of "live" testing for the happy path of their code.
The goals for such a boto3 emulation module should have the following qualities,
- lightweight and lazy - the boto3 emulation layer only intends to emulate those methods of the relevant services that our module wraps for a happy path. I don not intend to emulate errors for the first few iterations. If more services/methods are needed, it can be implemented as we go along.
- transparent - any boto3 client which imports the original boto3 module should not be aware that they're interacting with an emulation layer.
With this context, I have questions on,
- How to hot patch the boto3 imports? - Suppose I have an emulated module available, how should I go about dynamically using boto3 emulation layer over the original boto3 when not running in an AWS environment? Should I be hooking Python's module imports to redirect importing the original boto3 imports? It is obvious that I merely cannot use Python mocks.
- Any pitfalls I should be aware of when implementing such an emulation layer or does it exist already elsewhere?