PyTorch: Inconsistent behavior in torch.arange

user_2571014 0
Viewed 77

I'm using Google Colab and when I run the following code:

from torch import tensor, arange

print( arange(0.0, 1.2, 0.2) )
print( arange(tensor(0.0), tensor(1.2), tensor(0.2)) )

I get the output:

tensor([0.0000, 0.2000, 0.4000, 0.6000, 0.8000, 1.0000])
tensor([0.0000, 0.2000, 0.4000, 0.6000, 0.8000, 1.0000, 1.2000])

This discrepancy also occurs with arange(0.0,1.1,0.1) but there is no discrepancy with arange(0.0,1.5,0.5).

Why does seemingly similar code give different results, and how can I anticipate when this will occur?

pytorch arrays
1 Answers

This is a numerical precision issue :(

By default, Python stores floating point numbers in double precision (aka float64), while PyTorch uses float32 by default.

If you try:

tensor(1.2).item()
tensor(1.2).dtype  # torch.float32

you'll get 1.2000000476837158, and arange with that number will be different from the one with 1.2. In this specific case, if you try:

import torch
from torch import tensor, arange

print(arange(tensor(0.0), tensor(1.2, dtype=torch.float64), tensor(0.2)))

you'll get what you expect, however even float64 will have some precision issue eventually. Anyway, as Python also uses double precision, you won't have this problem converting a scalar back and forth in this case.

I'm not sure if you can anticipate it.

user_42282750
Related