Linear regression with numpy.linalg.lstsq and NaN values

Viewed 349

I want to fit a function

f:(Weight,Length)-> Z

to a 2D surface using numpy.linalg.lstsq. My dataset Z is defined in a (Weight,Length) plane. However I have erased the values that are not relevant: All absolute values larger than 3.3 is set to NaN using:

Z=np.where(np.abs(Z)>3.3,np.NaN,Z)

I do not want the algorithm to work hard to get a match where it is not important, sacrificing precision where it actually matters.

I have rendered the values I want to fit for illustration. Btw. my model is a 2d polynom. surface plot of the dataset

I want to calculate least square roots using numpy. As I have NaN values, I have decided to mask them using:

Z = np.ma.array(Z, mask=np.isnan(Z)) # Use a mask to mark the NaNs

But the call

c, r, rank, s = np.linalg.lstsq(A, Z, rcond=None)

Still gives the known error

LinAlgError: SVD did not converge in Linear Least Squares

The code itself works if I do not set values to NaN.

How can I run the fit with NaN values, telling numpy to ignore those?

0 Answers
Related