Convert column suffixes from pandas join into a MultiIndex

user_6095474 0
Viewed 1244

I have two pandas DataFrames with (not necessarily) identical index and column names.

>>> df_L = pd.DataFrame({'X': [1, 3], 
                         'Y': [5, 7]})

>>> df_R = pd.DataFrame({'X': [2, 4], 
                         'Y': [6, 8]})

I can join them together and assign suffixes.

>>> df_L.join(df_R, lsuffix='_L', rsuffix='_R')

    X_L Y_L X_R Y_R
0   1   5   2   6
1   3   7   4   8

But what I want is to make 'L' and 'R' sub-columns under both 'X' and 'Y'.

The desired DataFrame looks like this:

>>> pd.DataFrame(columns=pd.MultiIndex.from_product([['X', 'Y'], ['L', 'R']]), 
         data=[[1, 5, 2, 6],
               [3, 7, 4, 8]])

    X       Y
    L   R   L   R
0   1   5   2   6
1   3   7   4   8

Is there a way I can combine the two original DataFrames to get this desired DataFrame?

pandas python
2 Answers

You can use pd.concat with the keys argument, along the first axis:

df = pd.concat([df_L, df_R], keys=['L','R'],axis=1).swaplevel(0,1,axis=1).sort_index(level=0, axis=1)

>>> df
   X     Y   
   L  R  L  R
0  1  2  5  6
1  3  4  7  8
user_66711760

For those looking for an answer to the more general problem of joining two data frames with different indices or columns into a multi-index table:

# Prepend a key-level to the column index
# https://stackoverflow.com/questions/14744068
df_L = pd.concat([df_L], keys=["L"], axis=1)
df_R = pd.concat([df_R], keys=["R"], axis=1)

# Join the two dataframes
df = df_L.join(df_R)

# Reorder levels if needed:
df = df.reorder_levels([1,0], axis=1).sort_index(axis=1)

Example:

# Data:
df_L = pd.DataFrame({'X': [1, 3, 5], 'Y': [7, 9, 11]})
df_R = pd.DataFrame({'X': [2, 4], 'Y': [6, 8], 'Z': [10, 12]})

# Result:
#    X        Y          Z
#    L    R   L    R     R
# 0  1  2.0   7  6.0  10.0
# 1  3  4.0   9  8.0  12.0
# 2  5  NaN  11  NaN   NaN

This also solves the special case of the OP with equal indices and columns.

df_L.columns = pd.MultiIndex.from_product([["L", ], df_L.columns])
user_33889620
Related