update column in dataset using apache spark in scala

Viewed 762

I have Titanic dataset store in datasets. I want to create new dataset from the existing dataset. Which will modify the sex column of titanic dataset to child if age of the person is less than 16 as follows.

def isChild(age:String):String={
  if(age.toDouble<16)
  {
    "Child"
  }else
  {
    age
  }
}

I am trying to create dataset using as follows:

titanic_df.na.drop.map(x=>isChild(x.getString(5))).show()

Any help as i want to modify 4th column of dataset based on age column 4 of the dataset and also handle the NULL values.

enter image description here

1 Answers
Related