We are trying to run HuggingFace Transformers Pipeline model in Paperspace (using its GPU).
The problem is that when we set 'device=0' we get this error:
RuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 15.90 GiB total capacity; 476.40 MiB already allocated; 7.44 MiB free; 492.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
And if we don't set 'device=0' then GPU doesn't work (which is OK because default option is not to use it).
How to make GPU work for these models in Paperspace?
The code is pretty simple so far:
from transformers import pipeline
classifier = pipeline("zero-shot-classification",
model="Sahajtomar/German_Zeroshot"
#,device = 0
)
On Google Colab with the same model and same code we didn't have this problem. We just set 'device=0' and we got GPU running perfectly.