I have huge a dataframe (38 milj rows):
df = pd.DataFrame({'I':[1,2,3,4], 'C':[80,160,240,80],
'F':[(1,2,3,4),(5,7,2),(9,6,2,5,7),(4,0,8,3,2)]})
C F I
0 80 (1, 2, 3, 4) 1
1 160 (5, 7, 2) 2
2 240 (9, 6, 2, 5, 7) 3
3 80 (4, 0, 8, 3, 2) 4
Now I would like to filter out the rows which contain the number 3 in 'F'
To give:
C F I
0 80 (1, 2, 3, 4) 1
3 80 (4, 0, 8, 3, 2) 4
Is there a high performant, low memory usage way to do this?
I tried np.equal((3), df['F'].values).all() but this obviously does not work