Delete bigquery table content with pandas-gbq

user_11486307 0
Viewed 1424

I'm deleting big query rows from a table, using the "pandas-gbq" library, which works fine.

However, since this is a "read" action, by default the whole table content is being fetched, and, I do not want it to occur, since that is unnecessary.

This is my current code below, any ideas about a way to perform a delete action without fetching the table as a df?

Thanks in advance.

Delete gbq table rows - Today and yesterday

sql = """
DELETE FROM `bla.bla.bla`
WHERE Day = '{today}' OR Day = '{yesterday}'
"""
sql = sql.format(today = curr_date, yesterday= prev_date)
pandas_gbq.read_gbq(sql, project_id=project_id, credentials=credentials)
pandas google-bigquery sql python
1 Answers

Why not use google-cloud-bigquery to invoke the query, which provides better access to the BQ API surface?

pandas_gbq by its nature provides only a subset to enable integration with the pandas ecosystem. See this document for more information about the differences and migrating between the two.

Here's a quick equivalent using the google-cloud-bigquery:

def do_the_thing():

    from google.cloud import bigquery

    bqclient = bigquery.Client()
    sql = """
        DELETE FROM `bla.bla.bla`
        WHERE Day = '{today}' OR Day = '{yesterday}'
    """

    query = bqclient.query(sql)
    print("started query as {}".format(query.job_id))
    # invoke result() to wait until completion
    # DDL/DML queries don't return rows so we don't need a row iterator
    query.result()
user_2124350
Related