I have a Scala Spark Dataset, ds and two functions isTypeA() and isTypeB(), which take rows in that Dataset and return whether or not that row should be classified as A or B respectively. They can both return true for the same row, in which case I want to classify that row as A. Finally, I want C to be the rows that are neither A or B. I would like to save this as 3 separate Datasets.
I can do this by using filter and calling the functions multiple times
val a = ds.filter(isTypeA(_))
val b = ds.filter(row => !isTypeA(row) && isTypeB(row))
val c = ds.filter(row => !isTypeA(row) && !isTypeB(row))
but is there a more efficient way to do it?