Find mean from duration but logs from same day be as one

user_10375183 0
Viewed 30

I have logs of users:

df = pd.DataFrame({'users' : [1,1,1,2],
            'dates' : ['2018-09-06','2018-09-06','2018-09-10','2018-09-10'],
              'duration' : [3,3,8,8]})

gives:

user dates      duration
1    2018-09-09    3
1    2018-09-09    3
1    2018-09-10    8
2    2018-09-10    8
           ...

Here is the tricky part:

As you can see, some users divide their work in two parts. Thus, user 1 in day 28 has done 6 hours but appears as 3 and 3.

Time for simple stats:

When we do the mean hours, anyone who has part their times, their values appear as extreme (the 3's mess with the mean).

I could use the mean. But I want to see any extreme values that someone may have as in a day and compare it with the median.

Example: Instead of 8 hours this user still has worked 6 hours which is an extreme value, taken as for a day.

Final: So I want to take any daily hours as one and later compare their daily hours with mean and median.

One solution may be:

df.groupby(['users','days'])['duration'].mean()

but still does by day and if do not do by day it has each from every day as independent which messes with the mean.

pandas data-analysis statistics group-by python
1 Answers

This can solve it:

df = pd.DataFrame({'users' : [1,1,1,2],
                'dates' : ['2018-09-06','2018-09-06','2018-09-10','2018-09-10'],
                  'duration' : [3,3,8,8]})


a = df.groupby(['users','dates'])['duration'].sum().reset_index()
a.groupby(['users'])['duration'].mean().reset_index()

First find the sum of each day by adding all the parts if exist and then maintain it as one for that day. Later, find the mean for each day since all the values are from each day.

Result:

    users duration
0   1     7
1   2     8
user_46606140
Related