I am running this HF model on a M1. In order to run it on GPU I've set the torch device to MPS - torch.device("mps") - but when I run the code this way, the following warning is thrown:
UserWarning: The operator 'aten::remainder.Tensor_out' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1663312411556/work/aten/src/ATen/mps/MPSFallback.mm:11.)
next_tokens = next_tokens % vocab_size
And the model outputs: ['aaaaaaaaaaaaaaa'], for any given input image.
Running on CPU - torch.device("cpu") - does return appropriate results, such as ['an advertisement for a magazine on a computer screen'].
Any idea why this is happening? I know some procedures are not yet implemented on MPS, but utilising the CPU on those specific operations should, still, yield the same results for the same input images, right?