When a field of type Timestamp is nullable, serialization fails(trace and schema below) and the error disappears after changing the field to mandatory. Please help understand the issue.
Error:
Caused by: org.apache.kafka.common.errors.SerializationException: Error serializing Avro message
Caused by: org.apache.avro.AvroRuntimeException: Unknown datum type org.joda.time.DateTime: 2011-06-07T00:00:00.000Z
at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:757)
at org.apache.avro.specific.SpecificData.getSchemaName(SpecificData.java:314)
at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:721)
at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:205)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:123)
at org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:87)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:156)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:118)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:101)
at io.confluent.kafka.serializers.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:53)
at io.confluent.kafka.streams.serdes.avro.SpecificAvroSerializer.serialize(SpecificAvroSerializer.java:65)
Schema
{
"name": "shipdate",
"type": [
"null",
{
"type": "long",
"connect.version": 1,
"connect.name": "org.apache.kafka.connect.data.Timestamp",
"logicalType": "timestamp-millis"
}
],
"default": null
}
ENVIRONMENT
- Java 1.8
- Apache maven 3.5.2
- Avro 1.8.2 also tested with 1.9.0-SNAPSHOT
- avro-maven-plugin
- kafka-streams 2.0.0-cp1