How to transpose big files and how to get data file in smaller size?

user_18441047 0
Viewed 32

I have this huge excel file with over 400000 rows and 20 columns. I need to transpose the table but I was unable to do it with excel, then I was unable to do it with pandas. So I done it in a way that I converted to csv file.

import pandas as pd

df = pd.read_excel('file.xlsx')
df.to_csv('file.csv')

Then I was able to do it with csv file to txt...

import pandas as pd

df = pd.read_csv("file.csv")
 
transposed_df =df.T

with open('transposed_file_from_csv.txt', 'w') as outfile:
    transposed_df.to_string(outfile)

But for some reason I got txt file with 1.5 GB, with my laptop I'm unable to open this huge file. Is there an option to get file with smaller size? or and other idea is more then welcome.

Thanks in advance?

pandas python
1 Answers

If the intention is to save the transposed dataframe as csv, then it's the same command as in the early part of your snippet:

transposed_df =df.T
transposed_df.to_csv('new_file.csv')
user_106935960
Related