The following shows a portion of dataframe I am working on:
>>> df = pd.DataFrame(
{'id': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20],
'trip_id': [11,15,15,15,15,15,15,15,15,15,15,16,16,16,16,20,35,40,40,40],
'session_id': [1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1],
'segment_id': [3, 1, 3, 5, 7, 9, 1, 3, 5, 7, 9, 1, 5, 7, 9, 1, 9, 0, 4, 6],
'trip_distance': [42.74,5921.07,5921.07,5921.07,5921.07,5921.07,5921.07,5921.07,
5921.07,5921.07,5921.07,3539.58,3539.58,3539.58,3539.58,
4959.94,71.3,6527.81,6527.81,6527.81],
'length': [28.03,253.72,479.86,500.85,365.0,489.88,270.05,318.93,11.6,0.34,
4.37,352.13,12.96,6.11,19.65,3947.12,7.39,818.88,1.18,354.66]}
)
>>> df.head()
id trip_id session_id segment_id trip_distance length
0 1 11 1 3 42.74 28.03
1 2 15 1 1 5921.07 253.72
2 3 15 1 3 5921.07 479.86
3 4 15 1 5 5921.07 500.85
4 5 15 1 7 5921.07 365.00
It basically represents dataset of users' trips, where a trip maybe be covered in multiple sessions. The trips are then split into segments.
So I would like to produce a statistical plots from this, such as:
histogram of number of segments per trip
cdf of trips having segment length>=50
Also trip to segment split results in lost of component (length) of trips. I would like to know the proportion of this lost by trips (like what proportion of the trips lost 500? In this last case, I proceeded like so:
df['trip-segments-sum'] = df.groupby(['trip_id'])['length'].transform(sum)
df['lost_distance'] = df['trip_distance'] - df['trip-segments-sum']
df.head()
id trip_id session_id segment_id trip_distance length trip-segments-sum lost_distance
0 1 11 1 3 42.74 28.03 28.03 14.71
1 2 15 1 1 5921.07 253.72 2694.60 3226.47
2 3 15 1 3 5921.07 479.86 2694.60 3226.47
3 4 15 1 5 5921.07 500.85 2694.60 3226.47
4 5 15 1 7 5921.07 365.00 2694.60 3226.47
I at this point, I am kind of confused, not sure how to proceed.
Can you please help me?