scala spark union of empty dataframe

Viewed 30

I'm trying to perform union of two dataframes. One or both of the dataframes can be empty, leading to error

Union can only be performed on tables with the same number of columns, but the first table has 0 columns and the second table has 16 columns

How can I gracefully perform the empty check on the dataframes. Normal if chain doesn't look "clean"

if (df1.isEmpty) {
    res = df2
} else if (df2.isEmpty) {
    res = df1
} else if (df1.isEmpty || df2.isEmpty) {
    // end
} else {
    res = df1.union(df2)
}
 
0 Answers
Related