I am planning to use SageMaker Serverless Inference in conjunction with SageMaker Neo to deploy my model for serverless, low latency inference. However, documentation is not clear whether it is possible to do so.
According to Instance Types for Neo here, I can use e.g. ml_m4 instance. However, OutputConfig for Neo Compiler here specifies that TargetDevice can be lambda. Serverless Inference docs specify that "Serverless Inference integrates with AWS Lambda to offer you high availability...", so I assume that underlying instances are of the same (or compatible) type with AWS Lambda.
- Have you used such combination? Does it work?
- Which instance type should I compile for with Neo?