Too many categories to pivot_table in python

Viewed 31

I would like to reshape the following table, recording the medications taken by each patient at each admission.

Admission ID Pills Amount
P001-001 M1 0.13
P001-001 M2 0.43
P001-001 M3 0.73
P002-001 M1 0.13
P002-001 M2 0.43
P002-001 M3 0.73

I would like to reshape the table as follow:

Pills M1 M2 M3
Admission ID
P001-001 0.13 0.43 0.73
P002-001 0.13 0.43 0.73

So, I used the following code to reshape the table.

pivot = pd.pivot_table(
    data=df,
    index='Admission ID',
    columns='Pills',
    values='Amount',
    aggfunc='mean',
    sort=True
)

However, Since my table is too large, which consists of 8k categories of pills and more than 1 billion of rows, I am not able to execute the above code to reshape the df due to running out of memory. I would like to ask if there are any other ways to reshape my table? Thank you.

0 Answers
Related