I would like to reshape the following table, recording the medications taken by each patient at each admission.
| Admission ID | Pills | Amount |
|---|---|---|
| P001-001 | M1 | 0.13 |
| P001-001 | M2 | 0.43 |
| P001-001 | M3 | 0.73 |
| P002-001 | M1 | 0.13 |
| P002-001 | M2 | 0.43 |
| P002-001 | M3 | 0.73 |
I would like to reshape the table as follow:
| Pills | M1 | M2 | M3 |
|---|---|---|---|
| Admission ID | |||
| P001-001 | 0.13 | 0.43 | 0.73 |
| P002-001 | 0.13 | 0.43 | 0.73 |
So, I used the following code to reshape the table.
pivot = pd.pivot_table(
data=df,
index='Admission ID',
columns='Pills',
values='Amount',
aggfunc='mean',
sort=True
)
However, Since my table is too large, which consists of 8k categories of pills and more than 1 billion of rows, I am not able to execute the above code to reshape the df due to running out of memory. I would like to ask if there are any other ways to reshape my table? Thank you.