I am using the AI platform of GCP. I need to choose the size of a boot disk and a data disk during the creation of the VM instance please see this attached image.
I have 500GB of data to be analysed in the AI platform. Should I choose more than 500GB disk size for both the boot disk and the data disk, OR is it enough to increase only one of the disk sizes (e.g., data disk size to be 500GB)? I am also not sure about the different roles between boot disk and data disk. Thank you.
It depends!!
You have the boot disk dedicated to the boot system. 100Gb is generally enough for this task
Data disk can be used to store your data. And it's depends on the volume of your data.
However, your data disk can store your data, it's not mandatory, it depends how you store your data. You can store your data on the boot disk. it's not a best practice because, most of the time, you can consider that the boot disk is dependent to the VM, the data not.
Imagine this process: you create a small VM, store your data in the data disk and create your model and training loop. When done, you delete your VM, create a bigger one (with GPUs for example) and attach your data disk. No need to reload the data, your model and so on, to train your model with this big VM
When you create a new Notebook instance using Notebooks API, the API will create a VM with 2 disks:
The reason to do this is to support Upgrade, where you can upgrade your VM by replacing the Boot disk with a new OS while preserving your data. To answer your question, I would allocate space in data disk.
