How to export more than 16000 rows in bigquery table to local machine as CSV file?

user_10853669 0
Viewed 5754

I want to export output of my select statement in Bigquery. There is "Save Results" option available but it has limit of 16,000 rows.

How can i export more than that?

google-bigquery google-cloud-platform
2 Answers

There is a quite easier step by using the export function. You can perform a simple query to export directly to GCS

EXPORT DATA OPTIONS(
  uri='gs://MyBucket/path/to/file*.csv',
  format='CSV',
  overwrite=true,
  header=true,
  field_delimiter=',') AS
SELECT * from MyTable WHERE condition.

However, it's greatly possible that several files are generated. And thus you have to get all of them with gsutil.

The advantage of Daniel's solution, is that you can choose to export the table to only one file.

user_113725930

If your data has more than 16,000 rows you'd need to save the result of your query as a BigQuery Table. BigQuery options

Afterwards, export the data from the table into Google Cloud Storage using any of the available options (such as the Cloud Console, API, bq or client libraries).

Finally, you can use any of the available methods in Google Cloud Storage (Cloud Console, API, gsutil or client libraries) to download the CSV file within your local environment.

Since for the time being you can't export data from a Table directly to a local file.

user_121632400
Related