How to pass a group of RelationalGroupedDataset to a function?

Viewed 5166

I am reading a csv as a Data Frame by below:

val df  = sqlContext.read.format("com.databricks.spark.csv").option("header", "true").load("D:/ModelData.csv")

Then I group by three columns as below which returns a RelationalGroupedDataset

df.groupBy("col1", "col2","col3")

And I want each grouped data frame to be send through the below function

 def ModelFunction(daf: DataFrame) = {

    //do some calculation

          }

For example if I have col1 having 2 unique (0,1) values and col2 having 2 unique values(1,2) and col3 having 3 unique values(1,2,3) Then i would like to pass each combination grouping to the Model function Like for col1=0 ,col2=1,col3=1 I will havea dataframe and I want to pass that to the ModelFunction and so on for each combination of the three columns.

I tried

df.groupBy("col1", "col2","col3").ModelFunction();

But it throw an error.

.

Any help is appreciated.

1 Answers
Related