I have multiple GPU devices and want to run a Pytorch on them. I have already tried MULTI-GPU EXAMPLES and DATA PARALLELISM in my code by
device = torch.device("cuda:0,1,2")
model = torch.nn.DataParallel(model, device_ids=[0, 1, 2])
model.to(device)
in my code. But the training is still performed on one GPU (cuda:0). In the shell, I have also selected GPUs using export CUDA_VISIBLE_DEVICES=0,1,2 before running my code.
Could anyone help me to solve this issue please?