Generate values in separate dataframe

user_10924836 0
Viewed 51

I trying to generate random data with Pandas.

Data is need to be stored in two columns. The first column needs to contain categorical variables (from Stratum_1 until Stratum_19) each of these stratums can contain a random number of values.

Second column needs to have data in the range between 1 to 180000000 with a standard deviation of 453210, a mean of 170000, and a number of rows 100000.

I try to

   categorical = {'name': ['Stratum_1','Stratum_2','Stratum_3','Stratum_4','Stratum_5','Stratum_6','Stratum_7','Stratum_8','Stratum_9',
    'Stratum_10','Stratum_11','Stratum_12','Stratum_13','Stratum_14','Stratum_15','Stratum_16','Stratum_17','Stratum_18','Stratum_19']}


desired_mean = 170000
desired_std_dev = 453210

df = pd.DataFrame(np.random.randint(0,180000000,size=(100000, 1)),columns=list('1'))

I tried with this code above but don't know how to implement categorical and numerical values together with desired mean and standard deviation. So can anybody help how to solve this problem and generate?

pandas numpy python
2 Answers

Try:

import numpy as np
categorical = {'name': ['Stratum_1','Stratum_2','Stratum_3','Stratum_4','Stratum_5','Stratum_6','Stratum_7','Stratum_8','Stratum_9',
    'Stratum_10','Stratum_11','Stratum_12','Stratum_13','Stratum_14','Stratum_15','Stratum_16','Stratum_17','Stratum_18','Stratum_19']}


desired_mean = 170000
desired_std_dev = 453210

df = pd.DataFrame({'num':np.random.normal(170000, 453210,size=(300000, 1)).reshape(-1), 'cat':np.random.choice(categorical['name'], 300000)})
df[(0<df['num'])&(df['num']<180000000)].sample(100000)

result:

enter image description here

user_72816750

I decided to use the gamma distribution to generate your desired sample after thinking that the given parameters are not suitable for the normal distribution.

Code

import numpy as np
import pandas as pd

# desired parameters
n_rows = 100000
lower, upper = 1, 180000000
mu, sigma = 170000, 453210

# amount of shift
delta = lower

# parameters for the gamma distribution 
shape = ((mu - delta) / sigma) ** 2
scale = sigma**2 / (mu - delta)

# Create a dataframe
categories = {'name': [f'Stratum_{i}' for i in range(1, 19 + 1)]}
df = pd.DataFrame(categories).sample(n=n_rows, replace=True).reset_index(drop=True)

# Generate samples along with your desired parameters
generator = np.random.default_rng()
while True:
    df['value'] = generator.gamma(shape=shape, scale=scale, size=n_rows) + delta
    if df.value.max() <= upper:
        break

# Show statistics
print(df.describe())

Output

value
count 100,000
mean 169,403 (Target: 170,000)
std 449,668 (Target: 453,210)
min 1
25% 39.4267
50% 5529.28
75% 105,748
max 9.45114e+06
user_180598790
Related