Python: Storing estimated coefficients from different regressions in a single matrix

user_5252949 0
Viewed 405

I have the below code which goes through my dataframe (df_main) and runs one regression (applying sm.OLS()) per each unique value of project_x. Thus, the number of regressions corresponds to the number of unique values in project_x.

Question: How can I store the estimated (exponentiated!) coefficients from all regressions that I ran in one single matrix with the below dimensions?

  • Number of rows=number of unique project_x values
  • Number of columns=number of regression coefficients estimated

The matrix would therefore have the unique project_x values as rows and the regression coefficients estimated as columns with the intersection of each being the corresponding estimated regression coefficient.

My code:

import statsmodels.api as sm

for x in df_main.project_x.unique():
    df_holder=df_main[df_main.project_x == x]
    X = df_holder.drop(['unneeded1', 'unneeded2','unneeded3'], axis=1)
    X['constant']=1 
    Y = df_holder['sales']

    eq=sm.OLS(y, X)
    results=eq.fit()
    
    exp_coefficients=np.exp(results.params)
    print(exp_coefficients)
pandas numpy python
1 Answers

You can store exp_coefficients in a list in your loop and then convert into a matrix

Optionally you can also put it into a dataframe, see code ralated to df_index, df_columns, etc

import statsmodels.api as sm

df_index = []
coef_mtr = [] # start with an empty list
for x in df_main.project_x.unique():

    ...    # the rest of your code here
    exp_coefficients=np.exp(results.params)
#   print(exp_coefficients)
    coef_mtr.append(exp_coefficients)
    df_index.append(x)

coef_mtr = np.array(coef_mtr)

# create a dataframe with this data
df_columns = [f'coef_{n}' for n in range(coef_mtr.shape[1])]
df_matrix=pd.DataFrame(data = coef_mtr, index = df_index, columns = df_columns)
user_145514260
Related