How to merge one numpy array onto multiple dataframes

user_11666502 0
Viewed 262

I have a bunch of data frames. They all have the same columns but different amounts of rows. They look like this:

df_1 
   0
0  1
1  0
2  0
3  1
4  1
5  0

df_2
   0
0  1
1  0
2  0
3  1

df_3
   0
0  1
1  0
2  0
3  1
4  1

I have them all stored in a list.

Then, I have a numpy array where each item maps to a row in each individual df. The numpy array looks like this:

[3 1 1 2 4 0 6 7 2 1 3 2 5 5 5]

If I were to pd.concat my list of dataframes, then I could merge the np array onto the concatenated df. However, I want to preserve the individual df structure, so it should look like this:

   0  1
0  1  3
1  0  1
2  0  1
3  1  2
4  1  4
5  0  0

   0  1
0  1  6
1  0  7
2  0  2
3  1  1

   0  1
0  1  3
1  0  2
2  0  5 
3  1  5
4  1  5
pandas numpy python
1 Answers

Considering the given dataframes & array as,

df1 = pd.DataFrame([1,0,0,1,1,0])
df2 = pd.DataFrame([1,0,0,1])
df3 = pd.DataFrame([1,0,0,1,1])

arr = np.array([3, 1, 1, 2, 4, 0, 6, 7, 2, 1, 3, 2, 5, 5, 5])

You can use numpy.split to split an array into multiple sub-arrays according to the given dataframes. Then you can append those arrays as columns to their respective dataframes.

Use:

dfs = [df1, df2, df3]

def get_indices(dfs):
    """
    Returns the split indices inside the array.
    """
    indices = [0]
    for df in dfs:
        indices.append(len(df) + indices[-1])
    return indices[1:-1]

# split the given arr into multiple sections.
sections = np.split(arr, get_indices(dfs))
for df, s in zip(dfs, sections):
    df[1] = s # append the section of array to dataframe
    print(df)

This results:

# df1
   0  1
0  1  3
1  0  1
2  0  1
3  1  2
4  1  4
5  0  0

#df2
   0  1
0  1  6
1  0  7
2  0  2
3  1  1

# df3
   0  1
0  1  3
1  0  2
2  0  5
3  1  5
4  1  5
user_128331660
Related