Safely delete conda files from AWS deep learning AMI

user_841830 0
Viewed 421

I started up "Deep Learning AMI (Ubuntu 18.04) Version 27.0"; it comes with a 90GB disk, which seemed plenty, but over 60GB of that was already used. I only need python3, pytorch, cuda.

I found 30GB was inside ~/anaconda3/envs:

3.9G    aws_neuron_mxnet_p36
2.2G    aws_neuron_pytorch_p36
1.9G    aws_neuron_tensorflow_p36
2.5G    chainer_p27
1.2G    chainer_p36
2.1G    mxnet_p27
2.1G    mxnet_p36
729M    python2
866M    python3
2.5G    pytorch_p27
2.6G    pytorch_p36
2.2G    tensorflow2_p27
2.1G    tensorflow2_p36
2.3G    tensorflow_p27
1.8G    tensorflow_p36
31G total

Is it safe to just rm, say, the tensorflow and mxnet directories?

conda env list gives the same list. Is it better to do e.g. conda env remove tensorflow2_p27.

Are there likely to be any side effects of removing those packages? Is there a way to make sure nothing else depends on them before removing them?

anaconda deep-learning amazon-web-services python
2 Answers

It was safe. I used the following commands (note the -n is needed), and Pytorch, used with Python3 and cuda continued to work.

conda env remove -n tensorflow2_p27
conda env remove -n tensorflow_p27
conda env remove -n mxnet_p27
conda env remove -n mxnet_p36
conda env remove -n chainer_p27
conda env remove -n chainer_p36

After each step it asked if it was okay to delete a bunch of packages.

That freed up about 10GB, which gave me the buffer I needed. (So I didn't experiment deleting some of the others, but I'm fairly sure all the aws_neuron ones could have gone too.)

user_8418300

If you have to do a bunch of uninstalling of stuff, odds are you are using the wrong AMI for you.

If you don't want these services, I would select a different AMI that doesn't include these and then in the "User Data" section (bottom of step 3 in the EC2 launch wizard) include bash code to install/configure the software you want.

You can also change the storage size in step 4 of the launch wizard.

user_52533760
Related