Replacing NaN value with zero, but zero is a value in angular data. Cannot replace NaN with other value either. Best workaround?

user2852630 0
Viewed 29

Issue I'm trying to solve has to do with data representation/preprocessing. I have a 2-D numpy array which has been populated with angle values between the interval [ 0 , 2pi ]. Only problem is, there are alot of instances of NaN values which I replaced with the value of 0.

In the end, i plan to use this data in a deep learning network, so you can imagine the issue this is causing during learning. The representative value of the NaN (the zero) is unfortunately equivilent to the angles 0 and 2pi. I can't just choose any other value for the NaN as this can become representative of an angle in my loss function as well.

Currently, I'm using the following loss: 2-2*cos(AngleGroundTruth-AnglePredict).

I would appreciate any advice in tackling this problem.

Edit: The spatial positions in the 2-D matrix matter, so I can't just remove the NaNs and resize the matrix. This importance on the spatial positions is why I replaced the NaNs with zero

data-preprocessing deep-learning python
1 Answers

Where are these NaN values coming from in the first place? If you don't want them in your data, why don't you remove them instead of replacing them with 0?

You can filter a numpy array like so. Notice that we are keeping all elements that are not NaN:

import math
import numpy as np

arr = np.array([0, 3.14, float("NaN"), 1.83, 6.28])

filter_arr = []

for element in arr:
    filter_arr.append(!math.isnan(element))

newarr = arr[filter_arr]

print(filter_arr)
print(newarr)

Articles for reference:

usero1pu0
Related