Unable to encode element with coder 'AvroCoder'

Viewed 1910

In GCP Dataflow I am returning a PCollection of following class from a DoFn.

import java.util.List;
import java.util.Map;

import org.apache.avro.reflect.Nullable;

import com.google.cloud.dataflow.sdk.coders.AvroCoder;
import com.google.cloud.dataflow.sdk.coders.DefaultCoder;
@DefaultCoder(AvroCoder.class)
public class MyClass{
  @Nullable
  public String id;
  @Nullable
  public String foo;
  @Nullable
  public List<Integer> foo2;
  @Nullable
  public Map<String,List<String>> foo3;
  @Nullable
  public List<String> foo4;
}

Getting error:

Unable to encode element MyClass with coder 'AvroCoder'.

Full Strack Trace

The root cause looks to be this portion of the stack trace:

Caused by: java.lang.ClassCastException:
  java.lang.String cannot be cast to java.lang.Number
  at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:78)

My DoFn function returns above class's instances in a HashMap, like below:

 public static class Prep extends DoFn<TableRow, KV<String, MyClass>> {
} 
0 Answers
Related