Spark Dataset — Avoid duplicating Scala case class fields

Viewed 195

It is often the case that I want to type two Spark Dataset with very close columns like so:

case class Car (
  serial_number: Int,
  year: Int,
  brand: String,
  colour: String,
  horsepower: Int,
  weight: Int,
  width: Float,
  length: Float,
  seats: Int,
  owner: String,
  mileage: Int
)

case class CarWithOptionalMileage (
  serial_number: Int,
  year: Int,
  brand: String,
  colour: String,
  horsepower: Int,
  weight: Int,
  width: Float,
  length: Float,
  seats: Int,
  owner: String,
  mileage: Option[Int]        // <--- this is the only different field
)

I'm aware that a case class cannot extend another case class, however could there be a way to avoid all this field duplication and just specify the differing fields in the CarWithOptionalMileage?

0 Answers
Related