Extend Avro schema via Java API by adding one field

Viewed 5803

I use the Java API for Avro from Scala and wonder if there is an easy programmatical way to add a field to an existing record schema using the Avro GenericRecord / SchemaBuilder API?

3 Answers

Update

Alternatively, you can use SAvro.

libraryDependencies += "ca.dataedu" %% "savro" % "0.3.0"

and then

schema.addField("newField1", SchemaBuilder.builder().stringType())

More examples you can find in the README.

This is same answer but a different format of coding

@tmx has provided a complete answer. Once a schema is created, everything is locked. The only way is to implement a copy method. Here is a more compact version:

    // Start with a base schema 
    Schema base = ...;
    // Get a copy of base schema's fields.
    // Once a field is used in a schema, it gets a position.
    // We can't recycle a field and it will throw an exception.
    // Hence, we need a fresh field from each field of the old schema
    List<Schema.Field> baseFields = base.getFields().stream()
                .map(field -> new Schema.Field(field.name(), field.schema(), field.doc(), field.defaultVal()))
                .collect(Collectors.toList());
    // Add your field
    baseFields.add(new Schema.Field("Name", newFieldSchema));
    Schema newSchema = Schema.createRecord(
        base.getName(), 
        "New schema by adding a new field", 
        "com.my.name.space", 
        false, 
        baseFields);

having baseFields, you could do any modification you'd like, add/delete/modify.

Please don't forget to add aliases if you can have them

   List<Schema.Field> baseFields = base.getFields().stream()
            .map(field -> {
                Schema.Field f = new Schema.Field(field.name(), field.schema(), field.doc(), field.defaultVal());
                field.aliases().forEach(f::addAlias);
                return f;
            })
            .collect(Collectors.toList());
  
Related