I installed the huggingface Accelerate library to train the BERT model in multiple GPUs. I installed the Accelerate library using pip in the Anaconda environment. But I couldnt the accelerate command in powershell.
The following accelerate command
accelerate config
throws the following error:
accelerate : The term 'accelerate' is not recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ accelerate config
+ ~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (accelerate:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
I used the example mentioned in their GitHub Page. As the accelerate command was not working from poershell, I used the torch.distributed.launch to run the script as follows:
python -m torch.distributed.launch --nproc_per_node 1 --use_env ./nlp_example.py
Since I was using Windows OS, it gave the following error:
RuntimeError: Distributed package doesn't have NCCL built in
My doubt is, will it to possible to change the backend to use gloo, rather than 'NCCL' in Accelerate package?
Kindly help. Thank you.