How does numpy.random.choice work with replacement?

Viewed 815

The task is to choose 1000 clients randomly. In the dataset, the number of rows and unique IDs are the same.

I randomly select and then try to select the dataset with choosen IDs:

from numpy.random import choice
ids = choice(df.id, 1000)
df[df.id.isin(ids)]

The result is quite different: size of df[df.id.isin(ids)] is equal to 917. Why aren't there 1000 rows?

1 Answers

The default for np.random.choice() is to sample with replacement, replace=True.

Given a sample size of 1000, it's fairly likely that there will be a few repeats in the resulting sample.

E.g.

>>> import pandas as pd                                                                                                                                      
>>> import numpy as np                                                                                                                                       
>>> np.random.seed(444)                                                                                                                                      
>>> s = pd.Series(list(range(1000)))                                                                                                                         
>>> ids = np.random.choice(s, 1000)                                                                                                                          
>>> s[s.isin(ids)].count()                                                                                                                                   
623
>>> ids_no_rep = np.random.choice(s, 1000, replace=False)                                                                                                    
>>> s[s.isin(ids_no_rep)].count()                                                                                                                            
1000

If what you're looking to do is to shuffle the values of that column in-place, you can call np.random.shuffle(df["col"].values) for that column.

Related