I use the Java API for Avro from Scala and wonder if there is an easy programmatical way to add a field to an existing record schema using the Avro GenericRecord / SchemaBuilder API?
I use the Java API for Avro from Scala and wonder if there is an easy programmatical way to add a field to an existing record schema using the Avro GenericRecord / SchemaBuilder API?
Update
Alternatively, you can use SAvro.
libraryDependencies += "ca.dataedu" %% "savro" % "0.3.0"
and then
schema.addField("newField1", SchemaBuilder.builder().stringType())
More examples you can find in the README.
This is same answer but a different format of coding
@tmx has provided a complete answer. Once a schema is created, everything is locked. The only way is to implement a copy method. Here is a more compact version:
// Start with a base schema
Schema base = ...;
// Get a copy of base schema's fields.
// Once a field is used in a schema, it gets a position.
// We can't recycle a field and it will throw an exception.
// Hence, we need a fresh field from each field of the old schema
List<Schema.Field> baseFields = base.getFields().stream()
.map(field -> new Schema.Field(field.name(), field.schema(), field.doc(), field.defaultVal()))
.collect(Collectors.toList());
// Add your field
baseFields.add(new Schema.Field("Name", newFieldSchema));
Schema newSchema = Schema.createRecord(
base.getName(),
"New schema by adding a new field",
"com.my.name.space",
false,
baseFields);
having baseFields, you could do any modification you'd like, add/delete/modify.
Please don't forget to add aliases if you can have them
List<Schema.Field> baseFields = base.getFields().stream()
.map(field -> {
Schema.Field f = new Schema.Field(field.name(), field.schema(), field.doc(), field.defaultVal());
field.aliases().forEach(f::addAlias);
return f;
})
.collect(Collectors.toList());