How to obtain an Encoder[Seq[S]] when no implicit Encoder for S is availalbe?

Viewed 94

The implicit Encoders provided by Spark allow to obtain Encoders for sequences of primitive or product types quite easily, i.e.:

import spark.implicits._
val mySeqEncoder = newSequenceEncoder[Seq[Float]]
val myTupleSeqEncoder = newSequenceEncoder[Seq[(Float, Timestamp)]]

work without any problems.

But what if there is no implicit Encoder for the sequence type available? For example, suppose I need an Encoder for Seq[(Timestamp, Row)]. I may obtain an Encoder for each element using

import org.apache.spark.sql.Encoders

// schema = the schema of the row to encode
Encoders.tuple[Timestamp, Row](Encoders.TIMESTAMP, RowEncoder(schema))

Is there a way to build a sequence Encoder by giving an Encoder of the elements? Or is there another suitable approach?

0 Answers
Related