I'm following the example to read data from big query table. Below is my code.
According to the doc, the return value of parallel_read_rows() is tf.dataset. But dataset.cardinality().numpy() returned -2.
from tensorflow_io.bigquery import BigQueryClient
def read_bigquery(table_name):
bigquery_client = BigQueryClient()
project, dataset, table = table_name.split(".")
read_session = bigquery_client.read_session(
"projects/" + project,
project,
table,
dataset,
{
"label": {"output_type": dtypes.int64, "mode": BigQueryClient.FieldMode.NULLABLE},
"features": {"mode": BigQueryClient.FieldMode.REPEATED, "output_type": dtypes.float32},
},
requested_streams=2,
)
return read_session.parallel_read_rows()
dataset = read_bigquery(my_table)
Also, I got error when reading dataset.
returned ds type: <class 'tensorflow.python.data.ops.dataset_ops.ParallelInterleaveDataset'>
2022-06-30 15:18:47.292204: E tensorflow/core/framework/dataset.cc:577] UNIMPLEMENTED: Cannot compute input sources for dataset of type IO>BigQueryDataset, because the dataset does not implement `InputDatasets`.
2022-06-30 15:18:47.292243: E tensorflow/core/framework/dataset.cc:581] UNIMPLEMENTED: Cannot merge options for dataset of type IO>BigQueryDataset, because the dataset does not implement `InputDatasets`.
2022-06-30 15:18:47.292455: E tensorflow/core/framework/dataset.cc:577] UNIMPLEMENTED: Cannot compute input sources for dataset of type IO>BigQueryDataset, because the dataset does not implement `InputDatasets`.
2022-06-30 15:18:47.292482: E tensorflow/core/framework/dataset.cc:581] UNIMPLEMENTED: Cannot merge options for dataset of type IO>BigQueryDataset, because the dataset does not implement `InputDatasets`.
I'm using tensorflow-io==0.23.1