create a column in pyspark dataframe from values based on another dataframe

Viewed 55

I have two pyspark dataframes

df1 :

person_id   Name  serialNo  Maritalstatus  Location_name

 01         abc      10        M              America    
 02         xyz      13        S              London    
 03         def      14        M              Europe    
 04         qwe      15        M              Australia
 05         asd      16        M              Europe
 06         fgh      17        M              London
 07         aka      18        M              Australia
 08         fgi      19        M              London
 09         aba      20        M              Australia

df2:

Code   Location_Name    Location_Id

111        Australia          AUS    
112        America            USA    
123        London             UK    
124        Europe             EU

I want to add a column Location_Id to df1 fetching the matching IDs from df2, something like this below:

person_id   Name  serialNo  Maritalstatus  Location_name   Location_Id

 01         abc      10        M              America        USA
 02         xyz      13        S              London         UK 
 03         def      14        M              Europe         EU
 04         qwe      15        M              Australia      AUS
 05         asd      16        M              Europe         EU
 06         fgh      17        M              London         UK
 07         aka      18        M              Australia      AUS
 08         fgi      19        M              London         UK
 09         aba      20        M              Australia      AUS

How can I achieve this?

1 Answers

Simply join on Location_name

df1.join(df2, on='Location_name')
Related