Which is lower cost, Sagemaker or EC2?

I've listed some pros and cons from experience...

..., as opposed to marketing materials. If I were to guess, I'd say you have a much higher chance to experience all the drawbacks of SageMaker, than any one of the benefits.

Drawbacks

• Cloud vendor lock in: free improvements in the open source projects in the future and better prices in competitor vendors are difficult to get. Why don't AWS invest developers in JupyterLab, they have done limited work in open source. Find some great points here, where people have experienced companies using as few AWS services as possible with good effect.
• SageMaker instances are currently 40% more expensive than their EC2 equivalent.
• Slow startup, it will break your workflow if every time you start the machine, it takes ~5 minutes. SageMaker Studio apparently speeds this up, but not without other issues. This is completely unacceptable when you are trying to code or run applications.
• SageMaker Studio is the first thing they show you when you enter SageMaker console. It should really be the last thing you consider.
  • SageMaker Studio is more limited than SageMaker notebook instances. For example, you cannot mount an EFS drive.I spoke to a AWS solutions architect, and he confirmed this was impossible (after looking for the answer all over the internet). It is also very new, so there is almost no support on it, even by AWS developers.
• Worsens the disorganised Notebooks problem. Notebooks in a file system can be much easier to organise than using JupyterLab. With SageMaker Studio, a new volume gets created and your notebooks lives in there. What happens when you have more than 1...
• Awful/ limited terminal experience, coupled with tedious configuration (via Lifecycle configuration scripts, which require the Notebook to be turned off just to edit these scripts). Additionally, you cannot set any lifecycle configurations for Studio Notebooks.
• SageMaker endpoints are limited compared to running your own server in an EC2 instance.
• It may seem like it allows you to skip certain challenges, but in fact it provides you with more obscure challenges that no one has solved. Good luck solving them. The rigidity of SageMaker and lack of documentation means lots of workarounds and pain. This is very expensive.

Benefits

These revolve around the SageMaker SDK (the Sagemaker console and SageMaker SDK) (please comment or edit if you found any more benefits)

• Built in algorithms (which you can easily just import in your machine learning framework of choice): I would say this is worse than using open source alternatives.
• Training many models easily during hyperparameter search YouTube video by AWS (a fast way to spend money)
• Easily create machine learning related AWS mechanical turk tasks. However, mturk is very limited within SageMaker, so youre better off going to mturk yourself.

My suggestion

If you're thinking about ML on the cloud, don't use SageMaker. Spin up a VM with a prebuilt image that has PyTorch/ TensorFlow and JupyterLab and get the work done.

You are correct about EC2 being cheaper than Sagemaker. However you have to understand their differences.

• EC2 provides you computing power
• Sagemaker (try to) provides a fully configured environment and computing power with a seamless deployment model for you to start training your model on day one

If you look at Sagemaker's overview page, it comes with Jupyter notebooks, pre-installed machine learning algorithms, optimized performance, seamless rollout to production etc.

Note that this is the same as self-hosting a EC2 MYSQL server and utilizing AWS managed RDS MYSQL. Managed services always appears to be more expensive, but if you factor in the time you have to spent maintaing server, updating packages etc., the extra 30% cost may be worth it.

So in conclusion if you rather save some money and have the time to set up your own server or environment, go for EC2. If you do not want to be bothered with these work and want to start training as soon as possible, use Sagemaker.

UPDATE 2022-Apr SageMaker instances are 24% more expensive on average than equivalent EC2 instances - source: @amirathi

UPDATE 2021-Oct The average premium cost has lowered from previous +30% to +20% meaning SageMaker is becoming cheaper over the years. Disclaimer: I'm only checking EU pricing.

UPDATE 2020-Nov It is no longer the case that SageMaker/EC2 (Training) cost ratio is +40%. As of 2020 it's closer to +30% though in depends on the instance type:

| SM Instance Type[1] | SM Cost[1] | EC2 Instance Type[2] | EC2 Cost[2] | Ratio |
| ------------------- | -----------| -------------------- | ----------- | ------|
| ml.p3.2xlarge       | 4.779      | p3.2xlarge           | 3.823       | 1.25  |
| ml.p3.8xlarge       | 18.35      | p3.8xlarge           | 15.292      | 1.20  |
| ml.p3.16xlarge      | 35.172     | p3.16xlarge          | 30.584      | 1.15  |
| ml.p2.xlarge        | 1.658      | p2.xlarge            | 1.326       | 1.25  |
| ml.p2.8xlarge       | 12.73      | p2.8xlarge           | 10.608      | 1.20  |
| ml.p2.16xlarge      | 24.398     | p2.16xlarge          | 21.216      | 1.15  |
| ml.g4dn.xlarge      | 0.921      | g4dn.xlarge          | 0.658       | 1.40  |
| ml.g4dn.2xlarge     | 1.316      | g4dn.2xlarge         | 0.94        | 1.40  |
| ml.g4dn.4xlarge     | 2.107      | g4dn.4xlarge         | 1.505       | 1.40  |
| ml.g4dn.8xlarge     | 3.808      | g4dn.8xlarge         | 2.72        | 1.40  |
| ml.g4dn.12xlarge    | 6.846      | g4dn.12xlarge        | 4.89        | 1.40  |
| ml.g4dn.16xlarge    | 7.616      | g4dn.16xlarge        | 5.44        | 1.40  |
| ------------------- | -----------| -------------------- | ----------- | ------|
|                     |            |                      | AVERAGE:    | 1.30  |

[1] SageMaker prices are USD, eu-central-1 on-demand per hour, for Training (not hosting). Source: https://aws.amazon.com/sagemaker/pricing/

[2] All EC2 prices are USD for eu-central-1 on-demand per hour. Source: https://aws.amazon.com/ec2/pricing/on-demand/

Source: https://docs.google.com/spreadsheets/d/1g1uMPQm48pRlKE6Vv1fYIKzMIxOaG-6Sa43U1y0GU_I/

EC2 is absolutely low cast for small use cases but for big use cases, the maintenance and enhancement cost will be more in future. Also it will be a lot of engineering work/cost when you try to implement such kind of features

1. Auto-scaling : add the instances on run-time as per the load. Distributing load and creating and maintaining such type of infrastructure will be very costly

2. Multi Model Server : If you want to merge multiple endpoints so you can use your infra on full potential will be not easy

3. Versioning and Data Management : If you want to version your model accurately, manage their source code with data, it will not be easy in EC2 instances

4. Model Training Cycle : If you want to crate automatic model training cycle based on data receive, you need to create complete workflow which is very easy in Sagemaker

5. Incremental Learning or Transfer Learning : If you want to do model learning or transfer learning kind things, it will be hard to maintain on EC2 and it will be costly too

6. Elastic Inference : To speed-up your model performance in case of deep learning and reduce latency, this is out of the box functionality, which is costly in case of EC2, both development and then running cost

7. DevOps Integration : Sagemaker is giving out of the box CLI feature for DevOps integration which you need to develop for EC2 instances

I still feel, for small applications, Sagemaker 2-3 times costly as it charge per hour but you can use Sagemaker for batch process like up the instance once, do all the predictions for your permutations, store it in database and use it for serving and for bigger applications, use it as real time predictor.

If the question is about cost, then EC2. Sagemaker instances are charged at 25% premium.

You need to decide on your usecase. If you want to build model and deploy as API, sagemaker might be a solution as it has this end to end.

Also, Sagemaker now is supporting distributed trainings on multi node which is not easy to setup if you provision EC2 Instances as you need to setup VPCs and Networking. Which is a nightmare on AWS.

If you just need cloud instance for training, go for EC2. Vendor Lock in brought up as a point.

I'm shamelessly talking about our product here. Apologies on that guys. We have https://netbook.ai/. Its sagemaker for any cloud. You can connect your cloud credentials and use it through this.