Calculate slope of a time series dataframe for each row

Viewed 79

I have a dataframe containing time series data on a per year basis.

Dates 2015 2016 2017 2018 2019 2020
Data1 130  120  160  150  230  250
Data2 120  150  140  170  180  190
.
.
.

I want to calculate the slope for each row of the dataframe using least square regression ( the x-coordinates are the year columns, and the y-coordinates are the data contained in each column) and also a standard error, both are defined as:

Slope formula

Standard error formula

where n denotes the number of data points (6 in the previous example), xbar is the average of the years, ybar average of each row (ie: ( 130 + 120 + ...)/6 for the first row).

So for each row 2 new columns would be appended one for the slope and one for the standard error.

I am trying to define a function that I then would use with the apply method on the original dataframe.

def slope_computer(data: pd.Series) -> float:
    #getting the dates ( in string type ) and cast them to float
    x_values = [float(value) for value in data.index]
    x_mean = sum(x_values)/len(x_values)

    #getting the actual data (that differs from one row to the other) as a list
    y_values = [y for y in data]
    ymean = sum([y for y in data])/len(data)

    numerator = sum([(x-xmean)*(y_ymean) for x,y in zip(x_values, y_values)])
    denominator = sum([(x-xmean)**2 for x in x_values])

    return numerator/denominator

Is there a more efficient way or built-in methods to alleviate the steps used ?

1 Answers

I think we can make the process more efficient. If you can provide the data or part of it for a better evaluation.

Depending on the size of the data we can convert it to int16, int32, use .map or numpy.

Related