Merge by looking up multiple column values

user_14665536 0
Viewed 151

I have the following 2 tables in the form of a dataframe in pandas;

this is a list of all possible combinations:

Table A:
   0    1    2
 +---+----+----+
0| A |None|None|
 +---+----+----+
1| B |None|None|
 +---+----+----+
2|...|    |    |
 +---+----+----+
3| A | C  | D  |
 +---+----+----+
4| B | C  | D  |
 +---+----+----+

These are the associated values of the variables:

Table B:
  0   1
 +---+---+
0| A | 5 |
 +---+---+
1| B | 2 |
 +---+---+
2| C | 7 |
 +---+---+
3| D | 4 |
 +---+---+

What I need is something like this:

   0    1    2   3
 +---+----+----+---+
0| A |None|None| 5 |
 +---+----+----+---+
1| B |None|None| 2 |
 +---+----+----+---+
2|...|    |    |   |
 +---+----+----+---+
3| A | C  | D  | 7 |
 +---+----+----+---+
4| B | C  | D  | 7 |
 +---+----+----+---+

Where column 3 is found by looking up each associated value from Table A, columns 0,1 and 2 in Table B, column 0 and returning the largest of those values.

for example: row 3 shows a combination of A,C and D. Therefore column 3 looks up A in Table B, a value of 5, then it looks up Cin Table B, a value of 7. Lastly it looks up D in Table B, a value of 4. Of those 3 numbers 7 is the largest so it returns that.

So far I have tried pandas.merge with no luck

Update: I tried this:

Final=df1.insert(3,column='min space',value=df1.join(df2.set_index(0),on=0).max())

but it only returned None and doesn't account for multiple columns in df1 and if I try to add multiple columns;[0,1,2] it tells me I need the same number of columns for each

pandas merge python-3.x python
4 Answers

you can convert all cells to number (by associated df), then get the column with max value of each row.

import pandas as pd
df = pd.DataFrame({                 # original df
    0:['A', 'B', 'A', 'B',],
    1:[None, None, 'C', 'C',],
    2:[None, None, 'D', 'D',],
    })
rdf = pd.DataFrame({                # associated values
    0:['A', 'B', 'C', 'D',],
    1:[5, 2, 7, 4,],
    })

tdf = df                            # copy the original df
rdf = rdf.set_index(0)[1]           # set index of rdf for next line
tdf = tdf[tdf.columns].replace(rdf) # replace all values in tdf by rdf
tdf[3] = tdf.max(axis=1)            # column[3] = max of each row
df[3] = tdf[3]                      # add column[3] to original df

print(df)

   0     1     2    3
0  A  None  None  5.0
1  B  None  None  2.0
2  A     C     D  7.0
3  B     C     D  7.0
user_76152600 
df1[3] = df2.merge(
    df1.stack(dropna=False).reset_index(0), how='outer'
).groupby('level_0').max()

Example:

>>> df1
     0    1    2
0    A  NaN  NaN
1    B  NaN  NaN
2  ...  NaN  NaN
3    A    C    D
4    B    C    D

>>> df2
   0  1
0  A  5
1  B  2
2  C  7
3  D  4

>>> df1.columns
Int64Index([0, 1, 2], dtype='int64')

>>> df2.columns
Int64Index([0, 1], dtype='int64')

>>> df2[1].dtype
dtype('int64')

df1[3] = df2.merge(
    df1.stack(dropna=False).reset_index(0), how='outer'
).groupby('level_0').max()

>>> df1
     0    1    2    3
0    A  NaN  NaN  5.0
1    B  NaN  NaN  2.0
2  ...  NaN  NaN  NaN
3    A    C    D  7.0
4    B    C    D  7.0
Admin1

Try this..

#!/usr/bin/env python3
import pandas as pd

A, B, C, D = 5, 2, 7, 4

df = pd.DataFrame({
    0: [A, B, None, A, B],
    1: [None, None, None, C, C],
    2: [None, None, None, D, D]
    })

df[3] = df.max(axis=1)

OUTPUT:

     0    1    2    3
0  5.0  NaN  NaN  5.0
1  2.0  NaN  NaN  2.0
2  NaN  NaN  NaN  NaN
3  5.0  7.0  4.0  7.0
4  2.0  7.0  4.0  7.0
user_151946480

Try with replace

dfA['out'] = dfA.replace(dict(zip(dfB[0],dfB[1]))).max(1)
dfA
Out[487]: 
   0     1     2  out
0  A  None  None  5.0
1  B  None  None  2.0
2  A     C     D  7.0
3  B     C     D  7.0
user_79645270
Related