Is there a Pandas method to join based on index and column?

user_81439 0
Viewed 102

I'm trying to create a new column in a Pandas DataFrame by extracting a value from another DataFrame's. For each index, it should use the column value that matches the existing DataFrames value. Here is a solution that works, but I'm looking for the best way to do it using Pandas.

import pandas as pd

dates = pd.date_range('2020-01-01', '2020-01-03', freq='d')
A = pd.DataFrame({
    'i': [1,2,3],
}, index=dates)
B = pd.DataFrame({
    1: [11, 12, 13],
    2: [21, 22, 23],
    3: [31, 32, 33],
}, index=dates)

# replace this with a more efficient method, avoiding for-loop and creating C
r = [B.loc[k, v] for k, v in A.i.items()]
C = pd.DataFrame({'B': r}, index=dates)
pd.merge(A, C, left_index=True, right_index=True)

expected result:

                i   B
    2020-01-01  1  11
    2020-01-02  2  22
    2020-01-03  3  33
pandas join python
2 Answers

If I understood you correctly:

# Reshape main df with index as (date, i), remove axis name
_A = A.set_index('i', append=True).rename_axis(index=lambda x: None)

# Reshape sub df with index as (date, i), name series (column) as 'B'
_B = B.stack().rename('B')

# perform left join on indices
_A.merge(_B, how='left', left_index=True, right_index=True)

Result: 

               B
2020-01-01 1  11
2020-01-02 2  22
2020-01-03 3  33

You can chain the whole set into one line, but I wouldn't recommend this monstrosity:

A.set_index('i', append=True).rename_axis(index=lambda x: None) \
    .merge(B.stack().rename('B'), how='left', left_index=True, right_index=True)
user_91833440

You can use lookup in this situation:

A['B'] = B.lookup(A.index, A['i'])
print(A)

Output:

            i   B
2020-01-01  1  11
2020-01-02  2  22
2020-01-03  3  33
user_63615310
Related