DataFrame.to_sql equivalent of using "RETURNING id" with psycopg2?

Viewed 464

When inserting data using psycopg2 I can retrieve the id of the inserted row using a RETURNING PostgreSQL statement:

import psycopg2

conn = my_connection_parameters()
curs = conn.cursor()

sql_insert_data_query = (
    """INSERT INTO public.data
        (created_by, comment)
        VALUES ( %(user)s, %(comment)s )
        RETURNING id; # the id is automatically managed by the database.
    """
)

curs.execute(
    sql_insert_data_query,
    {
        "user": 'me',
        "comment": 'my comment'
    }
)

conn.commit()
data_id = curs.fetchone()[0]

and that's great because I need this id to write other data to, e.g. an associative table.

But when having a large dictionary to write to PostgreSQL (which keys are the column identifiers), it's more convenient to rely on pandas' DataFrame.to_sql() method:

import pandas as pd
from sqlalchemy import create_engine

engine = create_engine('postgresql+psycopg2://', creator=my_connection_parameters)
df = pd.DataFrame(my_dict, index=[0]) # this is a "one-row" DataFrame, each column being created from the dict keys
df.to_sql(
    name='user_table',
    con=engine,
    schema='public',
    if_exists='append',
    index=False
)

but there is no direct way to retrieve the id PostgreSQL has created when this record was actually inserted.

Is there a nice and reliable workaround to get it?
Or should I stick with psycopg2 to write my large dictionary using a SQL query?

0 Answers
Related