Apache Avro compile failing due to generated variables missing dollar signs

Question

I am setting up an Apache Avro module to be used for serializing and deserializing messages to Kafka.

I've tried various Gradle plugins, and they are all resulting in different errors.

The org.betterplugin.avro plugin seems to get me the closest, as it generates the Java and protocol files. However, one of the generated Java files has errors due to a missing dollar sign on one of the generated variables.

build.gradle:

plugins {
    id "org.betterplugin.avro" version "0.19.2-SNAPSHOT"

    // Error: Unable to find resource '/org/apache/avro/compiler/specific/templates/java/classic/enum.vm'
    // id "com.bakdata.avro" version "1.0.1"

    // Error: Could not find method generateAvroProtocol()
    // id "com.github.davidmc24.gradle.plugin.avro-base" version "1.3.0"

    // Error: property 'outputDir' is missing an input or output annotation.
    // id "com.commercehub.gradle.plugin.avro" version "0.99.99"
}

group = 'com.example'
description = 'AVRO Library'

dependencies {
    implementation "org.apache.avro:avro:1.11.0"
}

generateAvroProtocol {
    source("src/main/resources/avro")
    outputDir = file("build/generated-main-avro-protocol")
}

generateAvroJava {
    source("src/main/resources/avro")
    outputDir = file("build/generated-main-avro-java")
}

The result looks like this:


  // Used by DatumReader.  Applications should not call.
  @SuppressWarnings(value="unchecked")
  public void put(int field$, java.lang.Object value$) {
    switch (field$) {
    case 0: EXAMPLE_A = value != null ? value$.toString() : null; break;
    case 1: EXAMPLE_B = value != null ? value$.toString() : null; break;
    case 2: EXAMPLE_C = value != null ? value$.toString() : null; break;
    default: throw new org.apache.avro.AvroRuntimeException("Bad index");
    }
  }

The value$ parameter is referenced in the null check without the dollar sign which fails to compile with cannot find symbol variable value.

This is generated from avro-compiler's record.vm template:

  // Used by DatumReader.  Applications should not call.
  @SuppressWarnings(value="unchecked")
  public void put(int field$, java.lang.Object value$) {
    switch (field$) {
#set ($i = 0)
#foreach ($field in $schema.getFields())
    case $i: ${this.mangle($field.name(), $schema.isError())} = #if(${this.javaType($field.schema())} != "java.lang.Object" && ${this.javaType($field.schema())} != "java.lang.String")(${this.javaType($field.schema())})#{end}value$#if(${this.javaType($field.schema())} == "java.lang.String") != null ? value$.toString() : null#{end}; break;
#set ($i = $i + 1)
#end
    default: throw new IndexOutOfBoundsException("Invalid index: " + field$);
    }
  }

The template uses the dollar sign, so I'm not sure how this is getting missed.

user_10435528 · Answer

If I use the plugin name as com.github.davidmc24.gradle.plugin.avro instead of com.github.davidmc24.gradle.plugin.avro-base, I'm able to generate the java and protocol files

build.gradle:

plugins {
    id 'java'
    id "com.github.davidmc24.gradle.plugin.avro" version "1.3.0"
}

repositories {
    mavenCentral()
}

dependencies {
    implementation "org.apache.avro:avro:1.11.0"
}

generateAvroJava {
    source("src/main/resources/avro")
    include("**/*.avsc")
    outputDir = file("build/generated-main-avro-java")
}

generateAvroProtocol {
    source("src/main/resources/avro")
    include("**/*.avdl")
    outputDir = file("build/generated-main-avro-java")
}

Generated avpr:

{
  "protocol" : "ExampleMessage",
  "namespace" : "example.avro",
  "types" : [ {
    "type" : "record",
    "name" : "ExampleMessage",
    "fields" : [ {
      "name" : "body",
      "type" : "string"
    } ]
  } ],
  "messages" : { }
}

Generated put method:

  // Used by DatumReader.  Applications should not call.
  @SuppressWarnings(value="unchecked")
  public void put(int field$, java.lang.Object value$) {
    switch (field$) {
    case 0: body = value$ != null ? value$.toString() : null; break;
    default: throw new IndexOutOfBoundsException("Invalid index: " + field$);
    }
  }

The $ signs are correct in the generated java files which are getting compiled into class files successfully

Apache Avro compile failing due to generated variables missing dollar signs

1 Answers