Shuffle a pandas dataframe by groups

user_8437049 0
Viewed 7609

My dataframe looks like this

sampleID  col1 col2
   1        1   63
   1        2   23
   1        3   73
   2        1   20
   2        2   94
   2        3   99
   3        1   73
   3        2   56
   3        3   34

I need to shuffle the dataframe keeping same samples together and the order of the col1 must be same as in above dataframe.

So I need it like this

sampleID  col1 col2
   2        1   20
   2        2   94
   2        3   99
   3        1   73
   3        2   56
   3        3   34
   1        1   63
   1        2   23
   1        3   73

How can I do this? If my example is not clear please let me know.

pandas shuffle dataframe python
3 Answers

I found this to be significantly faster than the accepted answer:

ids = df["sampleID"].unique()
random.shuffle(ids)
df = df.set_index("sampleID").loc[ids].reset_index()

for some reason the pd.concat was the bottleneck in my usecase. Regardless this way you avoid the concatenation.

user_25306740

Just to add one thing to @cs95 answer. If you want to shuffle by sampleID but you want to have your sampleIDs ordered from 1. So here the sampleID is not that important to keep. Here is a solution where you have just to iterate over the gourped dataframes and change the sampleID.

groups = [df for _, df in df.groupby('doc_id')]

random.shuffle(groups)

for i, df in enumerate(groups):
     df['doc_id'] = i+1

shuffled = pd.concat(groups).reset_index(drop=True)

        doc_id  sent_id  word_id
   0       1        1       20
   1       1        2       94
   2       1        3       99
   3       2        1       63
   4       2        2       23
   5       2        3       73
   6       3        1       73
   7       3        2       56
   8       3        3       34
user_72759260
Related