How can I split a pandas DataFrame into multiple dataframes?

user_10164953 0
Viewed 6482

I have a dataframe which consists of 231840 rows. I need to split it into 161 separate tables, each table containing 1440 rows, i.e. the first table contains the first 1440 rows, the second table contains the next 1440 rows and so on until I get 161 separate tables with the combined number of rows being 231840 rows. Any ideas?

pandas rows dataframe python split
2 Answers

You can use, np.array_split to split the dataframe:

import numpy as np

dfs = np.array_split(df, 161) # split the dataframe into 161 separate tables

Edit (To assign a new col based on sequential number of df in dfs):

dfs = [df.assign(new_col=i) for i, df in enumerate(dfs, 1)]
user_128331660

simply use

import numpy as np

df_list = np.array_split(df, 3) # replace 3 with the amount of rows you want

In you case you should switch 3 with df(len) // desired_row_amount. We use // to round the result to an integer.
Or go old school and use a for loop, something along the lines of:

rows = 100  # example number of rows
df_list = []  # list to store dfs

for i in range(len(df) // rows):
    if i == len(df) // rows:  # if this is the last part of the df
        df_list.append(df[i*rows:])  # append the dataframe rows left
    else:
# append with a dataframe which has the desired amount of rows
        df_list.append(df[i*rows:(i+1)*rows])
user_122082610
Related