How to select n chunks randomly from a dataset

user_11111375 0
Viewed 533

I have 100 chunks of samples at which each chunk has 100 elements. The elements in one chunk are 5 seconds apart; However the time difference between the elements from different chunks is 5 hours at least (not fixed time). My main goal is to pick 10 chunks from the dataset Randomly.

My dataset:

index     time                  value   
i       12-1-2019 05:22:43       12         }  Chunk 1
i+1     12-1-2019 05:22:48       13
i+2     12-1-2019 05:22:53       14
..
i+99  ...............................

i+x      12-1-2019 13:32:12      31         }  Chunk 2
i+x+1    12-1-2019 13:32:17      77
i+x+2    12-1-2019 13:32:22      16
..
i+x+99  ...............................

i+x      14-1-2019 23:45:43      14         }  Chunk 3
i+x+1    14-1-2019 23:45:48      57
i+x+2    14-1-2019 23:45:53      46
..
i+x+99  ...............................

Note: the chunks are ordered (I.e: Chunk 1 occurred before chunk 2 that occurred before chunk 3 and so on).

I would like to pick 10 chunk of 100 randomly. However, the order of the elements in the chunk is important and can't be shuffled.

pandas random python
1 Answers

If I got you right your first chunk goes from index 0 to index 99, the second chunk from index 99 to index 199 and so on.

If that's correct you can just get ten unique random numbers between 0 and 99 and iloc the appropriate rows from your dataset.

Assuming your data is in a pandas DataFrame:

[dataset.iloc[index*100:index*100 + 100] for index in random.sample(range(0,100),10)]

I didn't test the code because I don't have your data but the above codeline should give you a list with 10 chunks of length 100.

If the chunks in your list need to be ordered as in your original dataframe you can just sort the random numbers:

[dataset.iloc[index*100:index*100 + 100] for index in sorted(random.sample(range(0,100),10))]
user_74084620
Related