How to find the correlation between a group of values in a pandas dataframe column

user_4666912 0
Viewed 8810

I have a dataframe df:

ID    Var1     Var2
1     1.2        4
1     2.1        6
1     3.0        7
2     1.3        8
2     2.1        9
2     3.2        13

I want to find the pearson correlation coefficient value between Var1 and Var2 for every ID

So the result should look like this:

ID    Corr_Coef
1     0.98198
2     0.97073

update:

Must make sure all columns of variables are int or float

pandas dataframe python
4 Answers

Simple solution:

df.groupby('ID').corr().unstack().iloc[:,1]
user_12918090

Since both solutions did not work for me I will post one that allows to calculate the correlation between one column in different groups for python3. This is hopefully addressing the same issue @BKS ran into.

data = df[['date', 'group_id', 'var1']]
data_new = data.set_index(['date', 'group_id']).unstack(['group_id'])
final_df = pd.DataFrame(data_new.to_numpy(), columns=data_new.columns)
dfCorr = final_df.corr()
user_6749300
Related