Pandas rolling windows return stranger and stranger result (wrong) result when a bigger data is fed in

Viewed 95

I have a dataset that is obtained from df[:400] to test out if my issue is caused by a smaller test case.

Due to body word limitations, I've pasted the dataset to Pastebin at

PLEASE CHECK EDIT 1 (BELOW) FOR UPDATED CODE AND DATA https://pastebin.com/Tkd0JfbW

And I have been trying to get a rolling window of mean with parameter center = True (so that it is cumulative before taking a mean) and within 3s.

With below code

df_a = df[:100]
df_b = df_a.rolling('3s',center=True).sum()
print(df_b)

It yields

SAMPLE SET: 100 - PARTIAL-CORRECT (STRANGE RESULT STARTING ON: 2021-05-11 16:12:57.813827)

                            ACCEL_X  ACCEL_Y    ACCEL_Z
DATETIME                                               
2021-05-11 16:12:56.306402    152.0    240.0   572812.0
2021-05-11 16:12:57.328342   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.339296   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.351769   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.367998   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.383116   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.399212   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.408833   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.425007   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.441168   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.453493   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.465365   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.481998   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.493914   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.510983   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.522969   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.539639   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.551563   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.568165   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.580067   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.596677   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.608628   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.625287   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.641043   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.654077   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.670658   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.682621   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.699244   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.711213   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.727846   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.739800   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.756443   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.768392   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.785021   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.796967   -552.0   -828.0  1637392.0
2021-05-11 16:12:57.813827   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.825742   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.842576   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.854535   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.871136   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.883092   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.899754   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.911661   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.928301   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.940567   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.957303   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.969251   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.985828   -712.0   -860.0  1621000.0
2021-05-11 16:12:57.997759   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.013816   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.026814   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.042404   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.055419   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.071990   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.083915   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.100488   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.112375   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.128992   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.140941   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.161295   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.173860   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.190141   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.202595   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.218905   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.231610   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.247816   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.256673   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.272304   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.284254   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.300833   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.312773   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.329358   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.341409   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.358005   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.371318   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.386947   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.399596   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.416036   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.432101   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.444055   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.460660   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.472584   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.489196   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.501143   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.517769   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.530770   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.546321   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.559335   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.574884   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.587905   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.604468   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.616443   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.633109   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.645092   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.661702   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.673645   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.690233   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.702315   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.718970   -712.0   -860.0  1621000.0
2021-05-11 16:12:58.735606   -712.0   -860.0  1621000.0

The above yield result is partial-correct, which is still incorrect.

However, when I take a larger sample set (in my test set, df[:200]), it yields a total strange and in my understanding, is incorrect.

df_a = df[:200]
df_b = df_a.rolling('3s',center=True).sum()
print(df_b)

Yields

SAMPLE SET: 200 - INCORRECT

                            ACCEL_X  ACCEL_Y    ACCEL_Z
DATETIME                                               
2021-05-11 16:12:56.306402    152.0    240.0   572812.0
2021-05-11 16:12:57.328342  -1116.0  -1068.0  1735032.0
2021-05-11 16:12:57.339296  -1072.0  -1148.0  1751368.0
2021-05-11 16:12:57.351769  -1080.0  -1148.0  1767704.0
2021-05-11 16:12:57.367998  -1000.0  -1096.0  1784108.0
2021-05-11 16:12:57.383116   -988.0  -1292.0  1800568.0
2021-05-11 16:12:57.399212  -1116.0  -1076.0  1816712.0
2021-05-11 16:12:57.408833  -1124.0  -1244.0  1833360.0
2021-05-11 16:12:57.425007   -968.0  -1060.0  1849696.0
2021-05-11 16:12:57.441168   -852.0  -1052.0  1866120.0
2021-05-11 16:12:57.453493   -776.0  -1224.0  1882584.0
2021-05-11 16:12:57.465365   -688.0  -1008.0  1898888.0
2021-05-11 16:12:57.481998   -668.0  -1220.0  1915452.0
2021-05-11 16:12:57.493914   -528.0  -1076.0  1931688.0
2021-05-11 16:12:57.510983   -604.0  -1248.0  1948296.0
2021-05-11 16:12:57.522969   -604.0  -1248.0  1948296.0
2021-05-11 16:12:57.539639   -824.0  -1444.0  1981068.0
2021-05-11 16:12:57.551563   -824.0  -1488.0  1997764.0
2021-05-11 16:12:57.568165   -848.0  -1320.0  2013876.0
2021-05-11 16:12:57.580067  -1016.0  -1476.0  2030524.0
2021-05-11 16:12:57.596677   -832.0  -1432.0  2046664.0
2021-05-11 16:12:57.608628   -912.0  -1488.0  2063200.0
2021-05-11 16:12:57.625287   -840.0  -1444.0  2079500.0
2021-05-11 16:12:57.641043   -820.0  -1512.0  2095704.0
2021-05-11 16:12:57.654077   -764.0  -1652.0  2112072.0
2021-05-11 16:12:57.670658   -340.0  -1760.0  2128428.0
2021-05-11 16:12:57.682621   -636.0  -2040.0  2144976.0
2021-05-11 16:12:57.699244   -352.0  -2124.0  2161124.0
2021-05-11 16:12:57.711213   -384.0  -2220.0  2177532.0
2021-05-11 16:12:57.727846   -552.0  -2104.0  2193708.0
2021-05-11 16:12:57.739800   -548.0  -2116.0  2210032.0
2021-05-11 16:12:57.756443   -608.0  -2040.0  2226292.0
2021-05-11 16:12:57.768392   -388.0  -1984.0  2242608.0
2021-05-11 16:12:57.785021   -584.0  -1964.0  2258796.0
2021-05-11 16:12:57.796967   -572.0  -1860.0  2274928.0
2021-05-11 16:12:57.813827   -860.0  -2028.0  2275048.0
2021-05-11 16:12:57.825742  -1044.0  -1704.0  2291276.0
2021-05-11 16:12:57.842576   -912.0  -1796.0  2307732.0
2021-05-11 16:12:57.854535  -1128.0  -1752.0  2323988.0
2021-05-11 16:12:57.871136  -1068.0  -1648.0  2340412.0
2021-05-11 16:12:57.883092  -1220.0  -1648.0  2356892.0
2021-05-11 16:12:57.899754  -1036.0  -1596.0  2373160.0
2021-05-11 16:12:57.911661   -928.0  -1832.0  2389688.0
2021-05-11 16:12:57.928301   -948.0  -1760.0  2405984.0
2021-05-11 16:12:57.940567   -760.0  -2024.0  2422636.0
2021-05-11 16:12:57.957303   -784.0  -2036.0  2439084.0
2021-05-11 16:12:57.969251   -552.0  -2084.0  2455644.0
2021-05-11 16:12:57.985828   -644.0  -2228.0  2472200.0
2021-05-11 16:12:57.997759   -544.0  -2228.0  2488516.0
2021-05-11 16:12:58.013816   -624.0  -2476.0  2505384.0
2021-05-11 16:12:58.026814   -596.0  -2260.0  2521472.0
2021-05-11 16:12:58.042404   -596.0  -2408.0  2537972.0
2021-05-11 16:12:58.055419   -672.0  -2380.0  2554268.0
2021-05-11 16:12:58.071990   -752.0  -2392.0  2570512.0
2021-05-11 16:12:58.083915   -864.0  -2396.0  2586860.0
2021-05-11 16:12:58.100488   -752.0  -2184.0  2602860.0
2021-05-11 16:12:58.112375   -816.0  -2376.0  2619416.0
2021-05-11 16:12:58.128992   -796.0  -2332.0  2635416.0
2021-05-11 16:12:58.140941  -1012.0  -2356.0  2652056.0
2021-05-11 16:12:58.161295  -1064.0  -2292.0  2668212.0
2021-05-11 16:12:58.173860  -1064.0  -2392.0  2684472.0
2021-05-11 16:12:58.190141  -1072.0  -2380.0  2700716.0
2021-05-11 16:12:58.202595  -1032.0  -2340.0  2716816.0
2021-05-11 16:12:58.218905  -1260.0  -2432.0  2733416.0
2021-05-11 16:12:58.231610  -1072.0  -2348.0  2749624.0
2021-05-11 16:12:58.247816  -1116.0  -2336.0  2766104.0
2021-05-11 16:12:58.256673  -1136.0  -2316.0  2782448.0
2021-05-11 16:12:58.272304  -1092.0  -2356.0  2798808.0
2021-05-11 16:12:58.284254  -1092.0  -2356.0  2798808.0
2021-05-11 16:12:58.300833   -992.0  -2368.0  2831604.0
2021-05-11 16:12:58.312773   -992.0  -2368.0  2831604.0
2021-05-11 16:12:58.329358   -908.0  -2300.0  2864508.0
2021-05-11 16:12:58.341409  -1084.0  -2364.0  2881136.0
2021-05-11 16:12:58.358005  -1188.0  -2200.0  2897420.0
2021-05-11 16:12:58.371318  -1188.0  -2200.0  2897420.0
2021-05-11 16:12:58.386947  -1284.0  -2120.0  2930232.0
2021-05-11 16:12:58.399596  -1264.0  -2268.0  2946540.0
2021-05-11 16:12:58.416036  -1420.0  -2220.0  2963048.0
2021-05-11 16:12:58.432101  -1316.0  -2056.0  2979424.0
2021-05-11 16:12:58.444055  -1232.0  -2160.0  2995808.0
2021-05-11 16:12:58.460660  -1348.0  -1972.0  3011944.0
2021-05-11 16:12:58.472584  -1320.0  -2120.0  3028576.0
2021-05-11 16:12:58.489196  -1408.0  -1996.0  3044760.0
2021-05-11 16:12:58.501143  -1316.0  -2064.0  3061224.0
2021-05-11 16:12:58.517769  -1340.0  -2148.0  3077464.0
2021-05-11 16:12:58.530770  -1328.0  -2020.0  3093728.0
2021-05-11 16:12:58.546321  -1288.0  -2172.0  3110196.0
2021-05-11 16:12:58.559335  -1348.0  -2044.0  3126340.0
2021-05-11 16:12:58.574884  -1336.0  -2340.0  3142936.0
2021-05-11 16:12:58.587905  -1284.0  -2260.0  3159224.0
2021-05-11 16:12:58.604468  -1332.0  -2352.0  3175552.0
2021-05-11 16:12:58.616443  -1412.0  -2336.0  3192008.0
2021-05-11 16:12:58.633109  -1460.0  -2248.0  3208184.0
2021-05-11 16:12:58.645092  -1476.0  -2428.0  3224792.0
2021-05-11 16:12:58.661702  -1360.0  -2276.0  3241020.0
2021-05-11 16:12:58.673645  -1504.0  -2428.0  3257608.0
2021-05-11 16:12:58.690233  -1504.0  -2428.0  3257608.0
2021-05-11 16:12:58.702315  -1504.0  -2428.0  3257608.0
2021-05-11 16:12:58.718970  -1504.0  -2428.0  3257608.0
2021-05-11 16:12:58.735606  -1504.0  -2428.0  3257608.0
2021-05-11 16:12:58.747537  -1504.0  -2428.0  3257608.0
2021-05-11 16:12:58.764117  -1504.0  -2428.0  3257608.0
2021-05-11 16:12:58.776035  -1504.0  -2428.0  3257608.0
2021-05-11 16:12:58.792756  -1504.0  -2428.0  3257608.0
2021-05-11 16:12:58.805104  -1504.0  -2428.0  3257608.0
2021-05-11 16:12:58.820816  -1504.0  -2428.0  3257608.0
2021-05-11 16:12:58.837672  -1524.0  -2236.0  3241060.0
2021-05-11 16:12:58.849815  -1652.0  -2460.0  3224848.0
2021-05-11 16:12:58.861772  -1504.0  -2328.0  3208224.0
2021-05-11 16:12:58.878388  -1464.0  -2532.0  3192092.0
2021-05-11 16:12:58.890301  -1536.0  -2400.0  3175556.0
2021-05-11 16:12:58.906921  -1756.0  -2448.0  3159264.0
2021-05-11 16:12:58.918874  -1624.0  -2684.0  3142932.0
2021-05-11 16:12:58.935466  -1392.0  -2552.0  3126304.0
2021-05-11 16:12:58.948566  -1584.0  -2692.0  3110132.0
2021-05-11 16:12:58.964667  -1784.0  -2596.0  3093448.0
2021-05-11 16:12:58.977493  -1784.0  -2660.0  3077428.0
2021-05-11 16:12:58.993726  -1640.0  -2636.0  3060904.0
2021-05-11 16:12:59.008946  -1480.0  -2660.0  3044568.0
2021-05-11 16:12:59.025461  -1644.0  -2672.0  3011772.0
2021-05-11 16:12:59.034875  -1644.0  -2672.0  3011772.0
2021-05-11 16:12:59.051097  -1656.0  -2748.0  2995464.0
2021-05-11 16:12:59.065640  -1428.0  -2616.0  2978796.0
2021-05-11 16:12:59.078729  -1500.0  -2520.0  2962552.0
2021-05-11 16:12:59.091731  -1548.0  -2424.0  2946016.0
2021-05-11 16:12:59.107363  -1704.0  -2460.0  2929608.0
2021-05-11 16:12:59.123984  -1796.0  -2300.0  2913280.0
2021-05-11 16:12:59.135907  -1592.0  -2268.0  2896852.0
2021-05-11 16:12:59.148942  -1744.0  -2232.0  2880580.0
2021-05-11 16:12:59.165542  -1620.0  -2172.0  2863996.0
2021-05-11 16:12:59.181022  -1832.0  -2248.0  2847744.0
2021-05-11 16:12:59.194037  -1632.0  -2112.0  2831300.0
2021-05-11 16:12:59.210647  -1528.0  -2252.0  2815084.0
2021-05-11 16:12:59.222568  -1592.0  -2036.0  2798620.0
2021-05-11 16:12:59.239194  -1504.0  -2208.0  2782408.0
2021-05-11 16:12:59.251131  -1552.0  -2220.0  2766044.0
2021-05-11 16:12:59.267743  -1400.0  -2316.0  2749804.0
2021-05-11 16:12:59.279664  -1504.0  -2484.0  2733820.0
2021-05-11 16:12:59.291614  -1452.0  -2404.0  2717348.0
2021-05-11 16:12:59.308349  -1496.0  -2636.0  2701188.0
2021-05-11 16:12:59.320311  -1548.0  -2488.0  2684624.0
2021-05-11 16:12:59.336959  -1500.0  -2448.0  2668668.0
2021-05-11 16:12:59.353614  -1620.0  -2484.0  2652288.0
2021-05-11 16:12:59.365542  -1660.0  -2392.0  2635820.0
2021-05-11 16:12:59.382113  -1716.0  -2540.0  2619712.0
2021-05-11 16:12:59.394037  -1628.0  -2328.0  2603064.0
2021-05-11 16:12:59.410613  -1464.0  -2444.0  2587036.0
2021-05-11 16:12:59.422547  -1396.0  -2300.0  2570260.0
2021-05-11 16:12:59.439295  -1340.0  -2336.0  2553988.0
2021-05-11 16:12:59.451270  -1180.0  -2344.0  2537604.0
2021-05-11 16:12:59.468010  -1140.0  -2208.0  2521068.0
2021-05-11 16:12:59.479952  -1180.0  -2304.0  2504636.0
2021-05-11 16:12:59.496579  -1252.0  -2120.0  2487880.0
2021-05-11 16:12:59.509585  -1364.0  -2268.0  2471832.0
2021-05-11 16:12:59.525104  -1348.0  -2216.0  2455124.0
2021-05-11 16:12:59.538133  -1360.0  -2272.0  2438788.0
2021-05-11 16:12:59.553660  -1572.0  -2168.0  2422276.0
2021-05-11 16:12:59.566676  -1644.0  -2048.0  2405840.0
2021-05-11 16:12:59.583269  -1668.0  -2132.0  2389520.0
2021-05-11 16:12:59.595235  -1572.0  -2104.0  2373040.0
2021-05-11 16:12:59.611982  -1848.0  -2048.0  2356840.0
2021-05-11 16:12:59.623907  -1856.0  -1916.0  2340384.0
2021-05-11 16:12:59.640686  -1964.0  -1976.0  2324200.0
2021-05-11 16:12:59.652628  -1816.0  -1916.0  2307748.0
2021-05-11 16:12:59.669193  -1560.0  -1988.0  2291536.0
2021-05-11 16:12:59.681163  -1560.0  -1912.0  2275228.0
2021-05-11 16:12:59.697752  -1324.0  -2012.0  2258920.0
2021-05-11 16:12:59.709644  -1352.0  -2072.0  2242612.0
2021-05-11 16:12:59.726264  -1060.0  -2008.0  2226276.0
2021-05-11 16:12:59.738190   -824.0  -2124.0  2210144.0
2021-05-11 16:12:59.754792   -812.0  -1868.0  2193528.0
2021-05-11 16:12:59.766719   -828.0  -1872.0  2177196.0
2021-05-11 16:12:59.786977   -904.0  -1824.0  2144468.0
2021-05-11 16:12:59.796734   -904.0  -1824.0  2144468.0
2021-05-11 16:12:59.812912  -1100.0  -1864.0  2111428.0
2021-05-11 16:12:59.827952  -1100.0  -1864.0  2111428.0
2021-05-11 16:12:59.837342  -1124.0  -1988.0  2095276.0
2021-05-11 16:12:59.856997  -1104.0  -1832.0  2078588.0
2021-05-11 16:12:59.872100  -1116.0  -1832.0  2045836.0
2021-05-11 16:12:59.881850  -1116.0  -1832.0  2045836.0
2021-05-11 16:12:59.898437  -1068.0  -1868.0  2029568.0
2021-05-11 16:12:59.910325  -1308.0  -2004.0  2013248.0
2021-05-11 16:12:59.926924  -1160.0  -1728.0  1996764.0
2021-05-11 16:12:59.938968  -1124.0  -1864.0  1980748.0
2021-05-11 16:12:59.955565  -1088.0  -1708.0  1964144.0
2021-05-11 16:12:59.967538  -1020.0  -1900.0  1947904.0
2021-05-11 16:12:59.984341   -972.0  -1744.0  1931416.0
2021-05-11 16:13:00.000928   -840.0  -1716.0  1915056.0
2021-05-11 16:13:00.013203   -860.0  -1760.0  1898740.0
2021-05-11 16:13:00.028715   -660.0  -1668.0  1882212.0
2021-05-11 16:13:00.041760   -732.0  -1844.0  1866168.0
2021-05-11 16:13:00.058355   -604.0  -1664.0  1849532.0
2021-05-11 16:13:00.070297   -500.0  -1740.0  1833324.0
2021-05-11 16:13:00.086912   -380.0  -1752.0  1816948.0
2021-05-11 16:13:00.098884   -552.0  -1616.0  1800476.0
2021-05-11 16:13:00.115439   -784.0  -1580.0  1784164.0
2021-05-11 16:13:00.127364   -728.0  -1464.0  1767676.0
2021-05-11 16:13:00.139304   -960.0  -1460.0  1751508.0
2021-05-11 16:13:00.155920   -860.0  -1320.0  1734884.0
2021-05-11 16:13:00.172499  -1076.0  -1392.0  1718556.0

You can see on ***SAMPLE SET: 200 - INCORRECT***, it yields a very different result as compared ***SAMPLE SET: 100 - PARTIAL-CORRECT***. within the same millisecond, it produces a different result. However, on SAMPLE SET: 100 -PARTIAL-CORRECT starting on 2021-05-11 16:12:57.813827) it differs result from the same seconds.

I've also tried to test on some small sets for instance df[100:200], it produces the result that I believe is partial-correct, when I perform df[80:200], it goes more and more buggy.

Is this because of my understanding of parameters that needs to be changed or tuning or is this a pure bug?

Link to pandas explanation of rolling function.

-------------------- Edit 1 --------------------

I read somewhere it may be caused by multi-index, hence I've removed the index column name, new pastebin link can be accessed via link via pastebin for your copy-paste ease.

Feel free to copy and paste my code and test it out via (Please use new link and copy to your clipboard before running)

import pandas as pd

a = pd.read_clipboard(sep='\s\s+')
a.index = pd.to_datetime(a.index)

Below is the code to test out one of the assumptions of the edging case:

Test Case 1: Full DF

df_test_1 = a.rolling('3s',center=True).sum()
df_test_1

Result: Messy Data [Very Messy]

Test Case 2: 50 Data

df_test_2 = a[:50].rolling('3s',center=True).sum()
df_test_2

Result: Strange Result, on 2021-05-11 16:12:57.796967 to 2021-05-11 16:12:57.813827, all the result jumps.

Test Case 3: 110 Data

df_test_3 = a[:110].rolling('3s',center=True).sum()
df_test_3

As of experimenting 100 to 110, data starts to go wild off. Which I believe is not related to edging-case but some more towards "misunderstanding" that pending enlightenment.

1 Answers

Your results are correct! To convince yourself take a random row, let's say 42, which corresponds to timestamp 2021-05-11 16:12:57.911661. Because you are using center=True that timestamp will be the center of your window. Now lets calculate the window bounds:

row = 42
window = pd.Timedelta('3s')
center = df.index[row]
start = center - window / 2
end = center + window / 2

Now we can add the rows like so:

In [536]: df[start:end].sum()
Out[536]:
ACCEL_X       -928
ACCEL_Y      -1832
ACCEL_Z    2389688
dtype: int64

We can verify that the rolling window gives the same result:

In [538]: df.rolling(window, center=True).sum().iloc[row]
Out[538]:
ACCEL_X       -928.0
ACCEL_Y      -1832.0
ACCEL_Z    2389688.0
Name: 2021-05-11 16:12:57.911661, dtype: float64

You can repeat the same operations for a slice of your data, lets say the first 100 rows:

In [537]: df[:100][start:end].sum()
Out[537]:
ACCEL_X       -712
ACCEL_Y       -860
ACCEL_Z    1621000
dtype: int64

In [539]: df[:100].rolling(window, center=True).sum().iloc[row]
Out[539]:
ACCEL_X       -712.0
ACCEL_Y       -860.0
ACCEL_Z    1621000.0
Name: 2021-05-11 16:12:57.911661, dtype: float64
Related