When I join two dataframes using Seq(col), I still get multiple columns when using df.*
E.g: '''
val schema = StructType( Array(
StructField("language", StringType,true),
StructField("users", StringType,true)
))
val rowData= Seq(Row("Java", "20000"),
Row("Python", "100000"),
Row("Scala", "3000"))
var dfFromData3 = spark.createDataFrame(rowData,schema)
val schema1 = StructType( Array(
StructField("language", StringType,true),
StructField("price", StringType,true)
))
val rowData1= Seq(Row("Java", "20"),
Row("Python", "10"))
var dfFromData4 = spark.createDataFrame(rowData1,schema1)
val combined = dfFromData3.join(dfFromData4,Seq("language"),"left")
'''
'''display(combined)''' - Has only one "language" column
but
'''display(combined.as("df").select("df.*"))''' - Has two "language" columns
Can someone please explain what is happening here?