apache beam method 'ReadFromBigQuery' producing TypeError: isinstance() arg 2 must be a type or tuple of types

Sahal Nurain 0
Viewed 26

I'm trying to read a csv file from GC but the read method is producing a 'typeError', below is a code snippet that I am working with:

import apache_beam as beam
from apache_beam.dataframe.convert import to_dataframe


def print_row(row):
    print(row)


pipeline = beam.Pipeline()


test_pipeline = (pipeline 
                  | "read_from_bigquery" >> beam.io.ReadFromBigQuery( table = 'gs://cloud-samples-data/bigquery/sample-transactions/transactions.csv')
                  | "print rows" >> beam.Map(lambda r: (print_row(r))))

pipeline.run()
apache-beam python
2 Answers

Solved. Here is the required change:

 "read_from_bigquery" >> beam.io.ReadFromText('gs://cloud-samples-data/bigquery/sample-transactions/transactions.csv')
Sahal Nurain0

For csv files you could also consider using the dataframe API, i.e.

import apache_beam as beam
from apache_beam.dataframe.convert import to_dataframe
from apache_beam.dataframe.io import read_csv


with beam.Pipeline() as p:
  pcoll = to_dataframe(p | read_csv('gs://cloud-samples-data/bigquery/sample-transactions/transactions.csv'))
robertwb0
Related