Python: Using Calculated Joint Probability Distributions on Existing Data to Create New, Randomly Generated Data

PineNuts0 0
Viewed 22

I created a sample dataset using the code below:

data = {'state': {7192: 'healthy', 7193: 'healthy', 7194: 'healthy', 7195: 'healthy', 7196: 'Non healthy', 7197: 'Non healthy', 7198: 'Non healthy'},
        'type':  {7192: 'W', 7193: 'A', 7194: 'W', 7195: 'W', 7196: 'Y', 7197: 'N', 7198: 'Y'}, 
        'tier': {7192: '1', 7193: '1', 7194: '1', 7195: '1', 7196: '3', 7197: '1', 7198: '1'}, 
        'color': {7192: 'red', 7193: 'blue', 7194: 'red', 7195: 'red', 7196: 'yellow', 7197: 'yellow', 7198: 'yellow'}
       }

enter image description here

From this dataframe, I've calculated a joint probability table in Python.

enter image description here

My question is: How do I randomly generate a new dataset based on these joint probabilities in code? The new data doesn't exactly have to match those joint distribution numbers. I want some randomness to peek through. I have been googling around but can't find anything.

probability dataframe python
1 Answers

IIUC, you can generate the dataset using numpy.random.choice:

import numpy as np
import pandas as pd

# combinations and probs from the probability table
d = np.array([
     ('healthy', 'W', 1, 'red'),
     ('Non healthy', 'N', 1, 'yellow'),
     ('Non healthy', 'Y', 1, 'yellow'),
     ('Non healthy', 'Y', 3, 'yellow'),
     ('healthy', 'A', 1, 'blue')
])

probs = [3/7, 1/7, 1/7, 1/7, 1/7]  # probs adding up to 1

# generate a 10-row array out of the 5 possible combinations above
res_arr = d[np.random.choice(d.shape[0], 10, p=probs), :]

res_df = pd.DataFrame(res_arr, columns=['state', 'type', 'tier', 'color'])
print(res_df)

Output:

         state type tier   color
0  Non healthy    N    1  yellow
1      healthy    W    1     red
2  Non healthy    Y    3  yellow
3      healthy    W    1     red
4  Non healthy    N    1  yellow
5      healthy    W    1     red
6  Non healthy    Y    1  yellow
7      healthy    W    1     red
8      healthy    W    1     red
9      healthy    W    1     red
userjgUF0
Related