big dataset, receiving "Unstacked DataFrame is too big, causing int32 overflow" when using unstack()

Viewed 2167

I tried pivot and groupby + unstack, both gave me errors. the error says

"Unstacked DataFrame is too big, causing int32 overflow".

I tried to downgrade pandas to 0.21 but its not compatible with python 3.7, so is there any solutions?

user_by_movie=  user_items.groupby(['user_id', 'movie_id'])['rating'].min().unstack()
1 Answers

Answer

This should work, let me know.

user_by_movie= user_items.groupby(['user_id', 'movie_id'])['rating'].min().to_frame()

Related