How convert a list with None in it to torch.tensor()?

Question

For example I have a variable z = [1, 3, None, 5, 6]

I would like to do: torch.tensor(z)

and get something like the following:

torch.tensor([1,3, None, 5,6], dtype=torch.float)

Howhever, such attempt raises an error

TypeError: must be real number, not NoneType

Is there a way to convert such list to a torch.tensor?

I don't want to impute this None value with something else. Numpy arrays are capable of converting such lists np.array([1, 3, None, 5, 6]), but I'd prefer not to convert back and forth variable either.

user_8763097 · Answer

I have a feeling that the tensor construction from data source doesn't allow the same kind of leniency that Numpy has with allowing None types. Please see also here for a discussion where others have asked about None types in tensors.

Looks like you're going to have to think about how to handle your missing data, either through imputation or another form of data cleaning.

Or perhaps a tensorshape is what you need.

user_5884955 · Answer

It depends on what you do. Probable the best is to convert None to 0.

Converting things to numpy arrays and then to Torch tensors is a very good path since it will convert None to np.nan. Then you can create the Torch tensor even holding np.nan.

import torch
import numpy as np

a = [1,3, None, 5,6]
b = np.array(a,dtype=float) # you will have np.nan from None
print(b) #[ 1.  3. nan  5.  6.]
np.nan_to_num(b, copy=False)
print(b) #[1. 3. 0. 5. 6.]
torch.tensor(b, dtype=torch.float) #tensor([1., 3., 0., 5., 6.])

Try also copy=True inside np.nan_to_num and you will get nan inside your tensor instead of 0.

How convert a list with None in it to torch.tensor()?

2 Answers