Pandas: how do you map a dictionary of dictionaries to 2 columns?

user_5490188 0
Viewed 400

I have the following dictionary:

rates = {'USD': 
              {'2019': 1,
               '2020': 2,
               '2021': 3},
         'CAD':
              {'2019': 4,
               '2020': 5,
               '2021': 6}
         }

and I have the following dummy dataframe:

   Item Currency Year Rate
0  1    USD      2019 
1  2    USD      2020
2  3    CAD      2021
3  4    CAD      2019
4  5    GBP      2020

I now want to populate the column Rate by mapping the correct rate, where rate = f(currency,year). I am trying with:

def map_rate(data, rates):

    for index, row in data.iterrows():

        currency = str(row['Currency'])

        if currency in list(rates.keys()):

            year = str(row['Year'])
            rate = rates[currency][year]

        else:
            rate = 1

    return rate

I use the above like the following:

df['Rate'] = map_rate(test, rates)

However, this is only returning the first rate, e.g. the value 1, instead of the appropriate rates:

    Item Currency Year  Rate
0   1    USD      2019  1
1   2    USD      2020  1
2   3    CAD      2021  1
3   4    CAD      2019  1
4   5    GBP      2020  1

The expected result is:

    Item Currency Year  Rate
0   1    USD      2019  1
1   2    USD      2020  2
2   3    CAD      2021  6
3   4    CAD      2019  4
4   5    GBP      2020  1

Where is my mistake?

pandas dataframe dictionary python
4 Answers

Using .apply

Ex:

df['Rate'] = df.apply(lambda x: rates[x['Currency']][x['Year']], axis=1)
# OR
df['Rate'] = df.apply(lambda x: rates.get(x['Currency'], dict()).get(x['Year'], 1), axis=1)
print(df)

Output:

  Item Currency  Year  Rate
0    1      USD  2019     1
1    2      USD  2020     2
2    3      CAD  2021     6
3    4      CAD  2019     4
4    5      GBP  2020     1
user_5323120

Here is one way, create a multiindex series from rates with stack, that you can reindex with the values from df to get the wanted rate per row.

df['rate'] = (
    pd.DataFrame(rates)
      .stack()
      .reindex(pd.MultiIndex.from_frame(df[['Year','Currency']].astype(str)), 
               fill_value=1)
     .to_numpy()
)
print(df)
   Item Currency  Year  rate
0     1      USD  2019     1
1     2      USD  2020     2
2     3      CAD  2021     6
3     4      CAD  2019     4
4     5      GBP  2020     1
user_92747320

Create another dataframe for rates

rates_df = pd.DataFrame(rates).unstack().reset_index()
rates_df.columns = ['Currency', 'Year', 'Rates']
rates_df['Year'] = rates_df['Year'].astype(int)

Then merge

df.merge(rates_df, on=['Currency', 'Year'], how='left').fillna(1)

Rates Dataframe

  Currency  Year  Rates
0      USD  2019      1
1      USD  2020      2
2      USD  2021      3
3      CAD  2019      4
4      CAD  2020      5
5      CAD  2021      6

Output

   Item Currency  Year  Rates
0     1      USD  2019    1.0
1     2      USD  2020    2.0
2     3      CAD  2021    6.0
3     4      CAD  2019    4.0
4     5      GBP  2020    1.0
user_51200490

This can easily be done with the built-in Pandas method df.apply(). Here's a more verbose example than the other posted answers.

Code:

def get_rate(row):
  if row['Currency'] in rates.keys():
    return rates[row['Currency']][row['Year']]
  else:
    return 1

df['Rate'] = df.apply(get_rate,axis=1)

print(df)
user_123107000
Related