How to assign random values from list to new column that doesn't exist in another column of the same row?

Viewed 449

I have a data set about 50k~ rows that has a certain Job ID and the User ID of the person that performed the job. It is represented by this sample I've created:

df = pd.DataFrame(data={
    'job_id': ['00001', '00002', '00003', '00004', '00005', '00006', '00007', '00008', '00009', '00010', '00011', '00012', '00013', '00014', '00015'],
    'user_id': ['frank', 'josh', 'frank', 'jessica', 'josh', 'eric', 'frank', 'josh', 'eric', 'jessica', 'jessica', 'james', 'frank', 'josh', 'james']
})


    job_id  user_id
0   00001   frank
1   00002   josh
2   00003   frank
3   00004   jessica
4   00005   josh
5   00006   eric
6   00007   frank
7   00008   josh
8   00009   eric
9   00010   jessica
10  00011   jessica
11  00012   james
12  00013   frank
13  00014   josh
14  00015   james

I wish to assign peer reviewers for those jobs in a new column called 'reviewer_id', where the reviewer is from the list of user_id's but the cannot be the same user_id. For example: frank can't review his own job, but jessica can.

My desired output would be something like this:

    job_id  user_id reviewer_id
0   00001   frank   jessica
1   00002   josh    frank
2   00003   frank   josh
3   00004   jessica eric
4   00005   josh    james
...
11  00012   james   frank
12  00013   frank   josh
13  00014   josh    eric
14  00015   james   eric

I'm quite new to python so I can only think of getting a list of unique user_id from reviewers = df['user_id'].unique().tolist() and iterating over the dataframe and assigning a reviewer ID but I know you should typically never iterate over a pandas dataframe. So I'm lost on how I would go about something like this.

7 Answers

The simplest way I can think of is to keep changing the reviewer until no one reviews their own works:

users = df['user_id'].unique()
df['reviewer_id'] = df['user_id']
self_review = lambda: df['reviewer_id'] == df['user_id']

while self_review().any():
    reviewers = np.random.choice(users, len(df))
    df['reviewer_id'] = df['reviewer_id'].mask(self_review(), reviewers)

In terms of performance, the code runs faster when there are more distinct users. Here's a faster version (requires Python 3.8 for the walrus := operator):

users = df['user_id'].unique()
df['reviewer_id'] = df['user_id']

while (self_review := df['user_id'] == df['reviewer_id']).any():
    reviewers = np.random.choice(users, self_review.sum())
    df.loc[self_review, 'reviewer_id'] = reviewers

You can use apply with set:

import random
unique_ids = set(df.user_id.unique())
assign = lambda x: random.choice(list(unique_ids - {x}))

df['reviewer_id'] = df.user_id.apply(assign)
print(df)

Output:

   job_id  user_id reviewer_id
0   00001    frank        eric
1   00002     josh        eric
2   00003    frank     jessica
3   00004  jessica       frank
4   00005     josh        eric
5   00006     eric     jessica
6   00007    frank        josh
7   00008     josh       frank
8   00009     eric       james
9   00010  jessica        eric
10  00011  jessica       frank
11  00012    james        josh
12  00013    frank     jessica
13  00014     josh     jessica
14  00015    james        eric

You can get the user ID values from the dataframe. Idea is, reshuffle the ID such a way that value can't be same as the original position value. Hence, the same user id is not going to get assign as reviewer ID.

You can shuffle the list using random.shuffle and zip the original User ID list and reshuffle the user ID list for checking the positional value.

import random

## shuffle the list
def make_index_shuffle(user_id):
    random_index = user_id[:]
    while True:
        random.shuffle(random_index)
        for index, index_value in zip(user_id, random_index):
            if index == index_value:
                break
        else:
            return random_index

## get the list of user ID values from the dataframe
user_id  = df.user_id.tolist()
## reshuffle the user ID such that the original ID of the list and reshuffled value should not be same
rearrange_id = make_index_shuffle(user_id)

df["reviewer_id" ] =  rearrange_id
df

You could use pandas apply to check 2 random reviewer choices against the value of the user, then return the first reviewer that's not the user.

import pandas as pd
from random import sample

personnel = df.user_id.unique().tolist()
def random_reviewer(x):
    reviewers = sample(personnel,2)
    if reviewers[0] == x['user_id']:
        return reviewers[1]
    return reviewers[0]
df['reviewer_id'] = df.apply(random_reviewer,axis=1)

Well you can always create a list from the column and then iterate over the list?

import pandas as pd 
import random 

user_list = []
reviewers = df['user_id'].unique().tolist() #unique names in user_id column
user_id_col = list(df['user_id']) #assign column to list

def rand_reviewer(list_of_reviewers):#function to generate rand user
    return list_of_reviewers[random.randint(0,4)]

for i in range(0, len(user_id_col)): #iterate over list ;)
    user_list.append(rand_reviewer(reviewers))
    while user_id_col[i] == user_list[i]: #generate random user until id's don't match
        user_list[i] = rand_reviewer(reviewers)

df['reviewer_id'] = user_list #add new column to df

The simplest way would be using apply and sample features of pandas as below:

df['reviewer_id'] = df.apply(lambda row: df[df['user_id']!=row['user_id']].sample()['user_id'].values[0], axis=1)

In the above line: df.where(df['user_id']!=row['user_id']) take all user ids except the current user and then using sample() and cleaning the result to simple string, we assign the results to new column of reviewer_id.

Note that this does not restrict sampling in anyway and one person can happen to take more review jobs than others as sampling is completely random and unconstrainted.

You can create a dictionary with the possible reviewers for each ID, and then use map to assign those possible reviewers for each row. So you get a list for each row, and you need to randomly select an element from each. I wasn't aware of a way to do this without a loop, but perhaps this is still reasonable:

unique = list(df['user_id'].unique())
conversion = {}

for u in unique:
    conversion[u] = [i for i in unique if i != u]

df['reviewer_id'] = [np.random.choice(i) for i in df['user_id'].map(conversion)]

Result:

   job_id  user_id reviewer_id
0   00001    frank       james
1   00002     josh        eric
2   00003    frank        josh
3   00004  jessica       james
4   00005     josh     jessica
...
...
...
Related