goavro and original Go data structure

Question

How can I recreate original data structure in Golang serialized with avro using goavro?

With this library https://github.com/hamba/avro it's quite easy.

out := SimpleRecord{}
err = avro.Unmarshal(schema, data, &out)

type of variable out is SimpleRecord.

Let's say I have this struct and avro schema:

type SimpleRecord struct {
    F1           int      `avro:"f1"`
    F2           string   `avro:"f2"`
    F3           string   `avro:"f3"`
    Dependencies []string `avro:"dependencies"`
}

func main() {
    avro_schema_txt := `{
        "type": "record",
        "name": "AvroData",
        "namespace": "data.avro",
        "doc": "docstring",
        "fields": [
            {
                "name": "f1",
                "type": "int"
            },
            {
                "name": "f2",
                "type": "string"
            },
            {
                "name": "f3",
                "type": "string"
            },
            {
                "name": "dependencies",
                "type": {
                    "type": "array",
                    "items": "string"
                }
            }
        ]
    }`
}

and then

codec, err := goavro.NewCodec(avro_schema_txt)
    if err != nil {
        log.Fatal(err.Error())
    }
    out, _, err := codec.NativeFromBinary(data)
    if err != nil {
        log.Fatal(err.Error())
    }
    fmt.Println(out)

where data is marshaled with avro, out is of type interface{}, so how can I "make" it SimpleRecord?

user_1533273 · Answer

There are two ways you can do it, either by casting out and doing a 'manual mapping' or by using codec.TextualFromNative. Both approaches are shown below for completeness,

Approach 1

Cast out from interface{} to map[string]interface{} and retrieve the values

...
simpleMap := out.(map[string]interface{})
f1 := simpleMap["f1"]
f2 := simpleMap["f2"] 
...

SimpleRecord {
    F1: f1,
    F2: f2,
    ...
}

Approach 2

Use TextualFromNative, the below code shows both encode decode process

var avro_schema_txt = `{
    "type": "record",
    "name": "AvroData",
    "namespace": "data.avro",
    "doc": "docstring",
    "fields": [
        {
            "name": "f1",
            "type": "int"
        },
        {
            "name": "f2",
            "type": "string"
        },
        {
            "name": "f3",
            "type": "string"
        },
        {
            "name": "dependencies",
            "type": {
                "type": "array",
                "items": "string"
            }
        }
    ]
}`

// added json to match field names in avro
type SimpleRecord struct {
    F1           int      `avro:"f1" json:"f1"`
    F2           string   `avro:"f2" json:"f2"`
    F3           string   `avro:"f3" json:"f3"`
    Dependencies []string `avro:"dependencies" json:"dependencies"`
}

func encodeDecode() {

    data := SimpleRecord{
        F1: 1,
        F2: "tester2",
        F3: "tester3",
        Dependencies: []string { "tester4", "tester5" },
    }

    codec, err := goavro.NewCodec(avro_schema_txt)
    if err != nil {
        log.Fatal(err.Error())
    }

    // encode

    textualIn, err := json2.Marshal(data)
    if err != nil {
        log.Fatal(err.Error())
    }

    nativeIn, _, err := codec.NativeFromTextual(textualIn)
    if err != nil {
        log.Fatal(err.Error())
    }

    binaryIn, err := codec.BinaryFromNative(nil, nativeIn)
    if err != nil {
        log.Fatal(err.Error())
    }

    // decode

    nativeOut, _, err := codec.NativeFromBinary(binaryIn)
    if err != nil {
        log.Fatal(err.Error())
    }

    textualOut, err := codec.TextualFromNative(nil, nativeOut)
    if err != nil {
        log.Fatal(err.Error())
    }

    var out = SimpleRecord{}
    err = json2.Unmarshal(textualOut, &out)
    if err != nil {
        log.Fatal(err.Error())
    }

    if !reflect.DeepEqual(data, out) {
        log.Fatal("should be equal")
    }
}

goavro and original Go data structure

1 Answers