Python - Match most recent date prior to another date (SQL)

Viewed 14

I am trying to match two dataframes using SQL in Python. I would like to match the most recent observation prior to a corresponding date on the other dataframe. So far, I have the following code, where I match all observations before the corresponding date (datadate).

In subsequent steps I create a variable that is the difference in days between filing_date and datadate, and then keep the observation with the least amount of days. However, this feels clunky and can present issues. Is there a way for me to do this within the SQL code? I would like to maintain all observations from datatable1. I included my current code and created tables below showing what I would like to do. Thanks for the help!

 import sqlite3
 sqlconn = sqlite3.connect(':memory:')
 datatable1.to_sql('datatable1',sqlconn, index=False)
 datatable2.to_sql('datatable2', sqlconn, index=False)

 qry = '''
SELECT DISTINCT
a.*, b.filing_date, b.value
FROM datatable1 AS A
LEFT JOIN datatable2 AS B
ON a.id = b.id AND b.filing_date < a.datadate
'''
compq_01 = pd.read_sql_query(qry, sqlconn)

datatable1:

id datadate
01 3/31/2015
01 6/30/2015
02 9/30/2015
03 12/31/2015
04 12/31/2015
05 12/31/2015

datatable2:

id filing_date value
01 2/15/2015 18
01 3/25/2015 12
01 6/28/2015 55
02 7/18/2015 67
02 8/25/2015 14
03 11/07/2015 18

What I want the resulting table to look like:

id datadate filing_date amount
01 3/31/2015 3/25/2015 12
01 6/30/2015 6/28/2015 55
02 9/30/2015 8/25/2015 14
03 12/31/2015 11/07/2015 18
04 12/31/2015 NaN NaN
05 12/31/2015 NaN NaN
0 Answers
Related