got error when reading data from big query table

Viewed 79

I'm following the example to read data from big query table. Below is my code. According to the doc, the return value of parallel_read_rows() is tf.dataset. But dataset.cardinality().numpy() returned -2.

from tensorflow_io.bigquery import BigQueryClient

 def read_bigquery(table_name):
        bigquery_client = BigQueryClient()
        project, dataset, table = table_name.split(".")
        read_session = bigquery_client.read_session(
            "projects/" + project,
            project,
            table,
            dataset,
            {
                "label": {"output_type": dtypes.int64, "mode": BigQueryClient.FieldMode.NULLABLE},
                "features": {"mode": BigQueryClient.FieldMode.REPEATED, "output_type": dtypes.float32},
            },
            requested_streams=2,
        )
        return read_session.parallel_read_rows()

dataset = read_bigquery(my_table)

Also, I got error when reading dataset.

returned ds type: <class 'tensorflow.python.data.ops.dataset_ops.ParallelInterleaveDataset'>
2022-06-30 15:18:47.292204: E tensorflow/core/framework/dataset.cc:577] UNIMPLEMENTED: Cannot compute input sources for dataset of type IO>BigQueryDataset, because the dataset does not implement `InputDatasets`.
2022-06-30 15:18:47.292243: E tensorflow/core/framework/dataset.cc:581] UNIMPLEMENTED: Cannot merge options for dataset of type IO>BigQueryDataset, because the dataset does not implement `InputDatasets`.
2022-06-30 15:18:47.292455: E tensorflow/core/framework/dataset.cc:577] UNIMPLEMENTED: Cannot compute input sources for dataset of type IO>BigQueryDataset, because the dataset does not implement `InputDatasets`.
2022-06-30 15:18:47.292482: E tensorflow/core/framework/dataset.cc:581] UNIMPLEMENTED: Cannot merge options for dataset of type IO>BigQueryDataset, because the dataset does not implement `InputDatasets`.

I'm using tensorflow-io==0.23.1

1 Answers

parallel_read_rows() returns iterator instead of dataset. In order to check value, you can use list(dataset.take(1).as_numpy_iterator()).

Related