- First create an
as_of_date column with pd.date_range in your dataframe that is a list of the dates between the start and end date per row with lambda x: (drop duplicates and keep last as well).
- Explode the dataframe on the
as_of_date in prepraration for a merge on date and port.
- Simply, merge the dataframes (Per your secondary question, you can simply exclude this step).
Step 1: Create Date Range Column
df['as_of_date'] = df.apply(lambda x: list(pd.date_range(x['start_date'], x['end_date'], freq='d')), axis=1)
df
Out[1]:
port currency start_date end_date \
0 PortA USD 2020-01-01 2020-01-04
1 PortA CAD 2020-01-04 2020-01-06
2 PortA EUR 2020-01-06 2020-01-11
3 PortA USD 2020-01-11 2020-01-15
as_of_date
0 [2020-01-01 00:00:00, 2020-01-02 00:00:00, 202...
1 [2020-01-04 00:00:00, 2020-01-05 00:00:00, 202...
2 [2020-01-06 00:00:00, 2020-01-07 00:00:00, 202...
3 [2020-01-11 00:00:00, 2020-01-12 00:00:00, 202...
Step 2: Explode the dataframe and drop duplicates
df = df.explode('as_of_date').drop_duplicates('as_of_date', keep='last')
df
Out[2]:
port currency start_date end_date as_of_date
0 PortA USD 2020-01-01 2020-01-04 2020-01-01
0 PortA USD 2020-01-01 2020-01-04 2020-01-02
0 PortA USD 2020-01-01 2020-01-04 2020-01-03
1 PortA CAD 2020-01-04 2020-01-06 2020-01-04
1 PortA CAD 2020-01-04 2020-01-06 2020-01-05
2 PortA EUR 2020-01-06 2020-01-11 2020-01-06
2 PortA EUR 2020-01-06 2020-01-11 2020-01-07
2 PortA EUR 2020-01-06 2020-01-11 2020-01-08
2 PortA EUR 2020-01-06 2020-01-11 2020-01-09
2 PortA EUR 2020-01-06 2020-01-11 2020-01-10
3 PortA USD 2020-01-11 2020-01-15 2020-01-11
3 PortA USD 2020-01-11 2020-01-15 2020-01-12
3 PortA USD 2020-01-11 2020-01-15 2020-01-13
3 PortA USD 2020-01-11 2020-01-15 2020-01-14
3 PortA USD 2020-01-11 2020-01-15 2020-01-15
Step 3: Merge the two dataframes (Per your secondary question - you can ignore this step if you don't have a tbl dataframe. Instead just run df = df[['port', 'as_of_date', 'currency']] to keep and reorder the columns that you need:
df_merge = pd.merge(df[['port', 'currency', 'as_of_date']], tbl, how='left', on=['as_of_date', 'port'])
df_merge
Out[3]:
port currency as_of_date
0 PortA USD 2020-01-01
1 PortA USD 2020-01-02
2 PortA USD 2020-01-03
3 PortA CAD 2020-01-04
4 PortA CAD 2020-01-05
5 PortA EUR 2020-01-06
6 PortA EUR 2020-01-07
7 PortA EUR 2020-01-08
8 PortA EUR 2020-01-09
9 PortA EUR 2020-01-10
10 PortA USD 2020-01-11
11 PortA USD 2020-01-12
12 PortA USD 2020-01-13
13 PortA USD 2020-01-14
14 PortA USD 2020-01-15
Full Code:
df = pd.DataFrame(data={
'port': ['PortA','PortA','PortA','PortA'],
'currency': ['USD', 'CAD', 'EUR', 'USD'],
'start_date': ['01/01/2020', '01/04/2020', '01/06/2020', '01/11/2020'],
'end_date': ['01/04/2020', '01/06/2020', '01/11/2020', '01/15/2020']
})
df[['start_date', 'end_date']] = df[['start_date', 'end_date']].apply(pd.to_datetime, errors='ignore')
tbl = pd.DataFrame(data={
'port': 'PortA',
'as_of_date': [x for x in pd.date_range(start='01/01/2020', end='01/15/2020')]
})
df['as_of_date'] = df.apply(lambda x: list(pd.date_range(x['start_date'], x['end_date'], freq='d')), axis=1)
df = df.explode('as_of_date').drop_duplicates('as_of_date', keep='last')
df_merge = pd.merge(df[['port', 'currency', 'as_of_date']], tbl, how='left', on=['as_of_date', 'port'])
df_merge