I have a Python script which is supposed to create quite a lengthy list, currently the VM I am using has an 8 core processor and 30GB RAM however that is not enough as the machine runs out of memory before it can generate the list. I am wondering, is it possible to have say 5 similar VMs and have them work on the same script and pool their resources together? I was thinking of using a MIG however I am wondering where would I then store the script and how would they be able to communicate among them?
1 Answers
Resolved this issue by changing my Python script to use a generator instead of saving the list to a var and simply for loop through that generator, no need for extra processing power.
Related