Subtracting a best fit line with numpy.polyfit()?

Question

So I'm working on a project and I have a set of data that I loaded in as a csv. The data has a spot that that I need to flatten out. I used the numpy.polyfit() function to find a line of best fit, but what I can't seem to figure out is how to subtract off the best fit line. Any advice?

Here is the code I'm using so far:

μ = pd.read_csv("C:\Users\ander\Documents\Data\plots and code\dataframe2.csv")
yvalue = "average"
xvalue = "xvalue"

X = μ[xvalue][173:852]
Y = μ[yvalue][173:852]

fit = np.polyfit(X, Y, 1)
μ = μ.subtract(fit, μ)

user_8103056 · Answer

The polyfit function finds the linear coefficient of the best fit. In order to subtract the line from your data, you first need to create the linear function itself. For example, you can use the numpy.poly1d function.

I'll show you an example. Since we don't have access to the .csv file I made up X and Y:

import matplotlib.pyplot as plt
import numpy as np

DATA_SIZE = 500
μ_X = np.sort(np.random.uniform(0,10,DATA_SIZE))
μ_Y = 3*np.exp(-(μ_X-7)**2) + np.random.normal(0,0.08,DATA_SIZE) + 0.5*μ_X

X = μ_X[50:200]
Y = μ_Y[50:200]

plt.scatter(μ_X, μ_Y, label='Full data')
plt.scatter(X, Y, label='Selected region')
plt.legend()
plt.show()

Now we can fit the baseline from the orange data and subtract the linear function from all the data (blue).

fit = np.polyfit(X, Y, 1)
linear_baseline = np.poly1d(fit) # create the linear baseline function
μ_Y = μ_Y - linear_baseline(μ_X) # subtract the baseline from μ_Y

plt.scatter(μ_X, μ_Y, label='Linear baseline removed')
plt.legend()
plt.show()

Subtracting a best fit line with numpy.polyfit()?

1 Answers