I have a problem. I would like to calculate the turnover for a customer in the last 6 months. The methods work on my dummy record, unfortunately the whole thing does not work on my real record as it is too slow. How can I rewrite this so that it performs faster?
Dataframe
customerId fromDate sales
0 1 2022-06-01 100
1 1 2022-05-25 20
2 1 2022-05-25 50
3 1 2022-05-20 30
4 1 2021-09-05 40
5 2 2022-06-02 80
6 3 2021-03-01 50
7 3 2021-02-01 20
Code
from datetime import datetime
from dateutil.relativedelta import relativedelta
import pandas as pd
def find_last_date(date_: datetime) -> datetime:
six_months = date_ + relativedelta(months=-6)
return six_months
def sum_func(row: pd.DataFrame, df: pd.DataFrame) -> int :
return df[
(df["customerId"] == row["customerId"])
& (row["fromDate"] + relativedelta(months=-6)<= df["fromDate"])
& (df["fromDate"] <= row["fromDate"])
]["sales"].sum()
d = {
"customerId": [1, 1, 1, 1, 1, 2, 3, 3],
"fromDate": [
"2022-06-01",
"2022-05-25",
"2022-05-25",
"2022-05-20",
"2021-09-05",
"2022-06-02",
"2021-03-01",
"2021-02-01",
],
"sales": [100, 20, 50, 30, 40, 80, 50, 20],
}
df = pd.DataFrame(data=d)
df["fromDate"] = pd.to_datetime(df["fromDate"], errors="coerce")
df["last_month"] = df["fromDate"].apply(find_last_date)
df["total_sales"]=df[["customerId", "fromDate"]].apply(lambda x: sum_func(x, df), axis=1)
print(df)
What I want
customerId fromDate sales last_month total_sales
0 1 2022-06-01 100 2022-03-01 200 # 100 + 20 + 50 + 30
1 1 2022-05-25 20 2022-02-25 100 # 20 + 50 + 30
2 1 2022-05-25 50 2022-02-25 100 # 50 + 20 + 30
3 1 2022-05-20 30 2022-02-20 30 # 30
4 1 2021-09-05 40 2021-06-05 40 # 40
5 2 2022-06-02 80 2022-03-02 80 # 80
6 3 2021-03-01 50 2020-12-01 70 # 50 + 20
7 3 2021-02-01 20 2020-11-01 20 # 20
print(df['customerId'].value_counts().describe())
count 53979.000
mean 87.404
std 1588.450
min 1.000
25% 2.000
50% 6.000
75% 22.000
max 205284.000
print(df['fromDate'].agg((min, max)))
min 2021-02-22
max 2022-03-26