I'm joining multiple dataframe and selecting specific columns to create desired output
For example
df1.join(df2, condition).join(df3, some condition).join(df4, some condition)
.map {x =>
val someData_1 = x._1._1._1
val someData_2 = x._1._1._2
val someData_3 = x._1._2
val someData_4 = x._2
Output(
someData_1.getAs("colName1").asInstanceOf[String],
.
.
.
someData_2.getAs("colName2").asInstanceOf[String],
.
.
.
someData_3.getAs("colName3").asInstanceOf[String],
.
.
.
someData_4.getAs("colName4").asInstanceOf[String],
.
.
.
)
}
As of now that's how I'm selecting specific columns from output of multiple DatFrames join and using asInstanceOf[String] as well to get String type.
Is there any other efficient way of selecting specific cols from multiple joins in DataFrame ? And secondly, how can I avoid using asInstanceOf[String] while creating new Instance Output using different columns ?
Please suggest ideas.
Thanks