I am trying to publish some data ( in avro schema with JSON encoding) to gcp pubsub and i have enable bigquery subscription so that data will auto route to BQ table.
Avro Schema:
{
"type": "record",
"name": "Sample",
"fields": [
{
"name": "abc",
"type": [
"null",
"string"
],
"default": null
}
]
}
BigQuery Table:
Sample_Table(abc: Integer marked as NULLABLE)
When below data is published to topic, then no data is written to BigQuery Table
{
"abc": {
"string": "testdata"
}
}
However, when i publish null values to topic as below, then null is inserted into BigQuery table.
{
"abc": null
}
Seems like BigQuery subscription is not recognizing the json structure internally?
Can someone please point me to something I am missing here?