How do I drop p rows every n rows of a pandas dataframe?

user_9559442 0
Viewed 58

I have a code which reads in data, however every 2000 rows there are about 10 header rows that I want to skip.

I have written the code below:

import pandas as pd
    
data = pd.read_csv('variance meting_2millsamples.lvm', delimiter= '\t',
names = ['t','V'], skiprows=11)

#data = data.drop(labels=range(0, 11), axis=0)
data = data.drop(labels=range(2000,2010), axis=0)
data = data.drop(labels=range(4010,4020),axis=0)
data = data.drop(labels=range(6020,6030),axis=0)
data = data.drop(labels=range(8030,8040),axis=0)
data = data.drop(labels=range(10040,8050),axis=0)
data = data.drop(labels=range(12050,12060),axis=0)
data = data.drop(labels=range(14060,12070),axis=0)
data = data.drop(labels=range(16070,12080),axis=0)

However my dataframe is about 14 million data points long. I've tried using a for loop but i couldn't get it to work. How could I make a for loop that automates the drops?

Is there a better way of ignoring/removing lines?

pandas dataframe for-loop python
2 Answers

This should do, set the limits in range(1,len(data)//2000) accordingly.

chain(*iterables) takes iterables as its parameters (hence the star operator in front of the list of ranges) and concatenates them.

from itertools import chain
cc = chain(*(range(2000*k+10*(k-1), 2010*k) for k in range(1,len(data)//2000)))
data = data.drop(labels=cc, axis=0)
user_47864400 
for i in range(0,data.shape[0],2000):
  data = data.drop(labels=range(i,i+10), axis=0)

you can do this although I ain't sure if its more efficient

user_103036850
Related