ASCII 0 Error on certain row when uploading to BigQuery from python

user_16390876 0
Viewed 161

I am doing an ETL from an API tot a Google BigQuery environment. While trying to upload a pandas dataframe to bigquery using pandas_gbq.to_gbq(), I've come across this error:

google.api_core.exceptions.BadRequest: 400 Error while reading data, error message: Error 
detected while parsing row starting at position: 0. Error: Bad character (ASCII 0) encountered.

After inspecting the row for which i've gotten this error, I could not find an ASCII symbol. I've only gotten this error for a few rows in the Dataframe, all other rows can be uploaded without an error. This makes me think that this is not a formatting error, but an illegal symbol in one of the field of the dataframe.

I've tried to remove illegal symbols using the string.replace function, however the error persists.

Is there a way for me to upload this data, either by cleaning it or reformatting it?

google-bigquery ascii python
1 Answers

You can use csvkit (csvkit is a suite of command-line tools for converting to and working with CSV, the king of tabular file formats) to clean your file, and, after that, you should be able to load it into BigQuery without a problem.

Also, here you can view how to successfully load a pandas DataFrame into BigQuery; it tells you how to properly load all the information from your DataFrame into BigQuery, what you need to do, and how to do it.

user_107828090
Related