I am trying to optimize a for loop that currently takes ~8 hours to run. I have recently learned about Vectorization, and would like to take advantage of this feature to speed up my code.
I have a dataframe (df) that has and index of days (1-250), and columns for each years. In my current code, I create a new empty dataframe (rdf) to hold the results of several metrics I need to calculate on this data.
In [155]: df
Out[155]:
Year 2007 2008 2009 2010 ... 2018 2019 2020 2021
Day ...
1 -0.380 -0.050 -0.050 -0.2100 ... -0.1150 -0.2950 -0.1200 -0.1150
2 -0.360 -0.030 -0.050 -0.1900 ... -0.1150 -0.3050 -0.1300 -0.1100
3 -0.360 -0.040 -0.050 -0.1950 ... -0.1100 -0.3050 -0.1350 -0.1100
4 -0.340 -0.040 -0.050 -0.1950 ... -0.1175 -0.3100 -0.1350 -0.1150
5 -0.350 -0.030 -0.055 -0.2000 ... -0.1200 -0.3100 -0.1450 -0.1200
... ... ... ... ... ... ... ... ... ...
246 -0.600 0.005 -0.530 0.0925 ... -0.1650 -0.1325 -0.1550 -0.0750
247 -0.700 0.020 -0.565 0.0900 ... -0.1875 -0.1075 -0.1500 -0.0700
248 -0.700 0.010 -0.615 0.0700 ... -0.2125 -0.0700 -0.1550 -0.0050
249 -0.700 -0.020 -0.630 0.1250 ... -0.2300 -0.0675 -0.1700 0.0025
250 -0.628 0.010 -0.420 0.1819 ... -0.2300 -0.0625 -0.1725 -0.0200
I then need to loop through each day in df (start day), and each possible end day (which is every day > the start day) to calculate the metrics.
count=0
for startDay in range(df.index.min(), df.index.max()+1):
for endDay in range(df.index.min()+1, df.index.max()+1):
inDatePrice = df.loc[df.index[starDay-1]]
peakAfterEntry = df.loc[df.index[starDay:endDay]].max()
inDatePriceToPeak = peakAfterEntry - inDatePrice
rdf.iloc[[count],0] = startDay
rdf.iloc[[count],1] = endDay
rdf.iloc[[count],2] = inDatePrice
rdf.iloc[[count],3] = inDatePriceToPeak
count+=1
Above is my for loop and an example of the metrics I need to calculate. There are 18 metrics total. Is there a way to vectorize these calculations?