Choose 2nd GPU on server

user_13100783 0
Viewed 1806

I am running code on server. There are 2 GPUs there, and the 1st one is busy. Yet, I can't find a way to switch between them. I am using pytorch if that is important. Following lines of code should be modified:

device = 'cuda' if torch.cuda.is_available() else 'cpu'

Modification may be stated only here. Thanks.

pytorch python-3.x
2 Answers

cuda by defaults chooses cuda:0, switching to the other GPU may be done through cuda:1

So, your line becomes:

device = 'cuda:1' if torch.cuda.is_available() else 'cpu'

You can read more about CUDA semantics.

user_73476310

Here is the way I'm doing it while using FastAI and pre-trained model for inference. First, while model definition with fai (import fastai.vision.all as fai) I obtain the model instance and put it to specified GPU (say with gpu_id=3):

model = fai.nn.Sequential(body, head)
model.cuda(device=gpu_id)

Then, while loading model weights I also specify which device to use (otherwise it creates the copy of a model in GPU 0):

model.load_state_dict(torch.load(your_model_state_filepath, torch.device(gpu_id)))
user_49490400
Related