Python -- Reduce list of duplicates but preserve duplicates in list if they are alternating

user_8193808 0
Viewed 47

I'm working with a Pandas dataframe, and I need to reduce a column's list of values while preserving alternating duplicates, if they exist, and while preserving order. I'm able to mask the values such that there are only ever two distinct values to work with (e.g., A and B below).

(It's best to show...) I'm looking to define the reduce_list() method below...

dummy_arr_one = ['A','A','B','B','A','A','A','A','B','B','B']
dummy_arr_two = ['A','A','A','B','B','B']
df = pd.DataFrame({"instance":
                  ["group_one" for x in range(0,len(dummy_arr_one))] + ["group_two" for y in range(0,len(dummy_arr_two))], 
                  "value":dummy_arr_one + dummy_arr_two
                 })

>> x = df[df['instance']=='group_one']['value'].values   # ['A','A','B','B','A','A','A','A','B','B','B']
>> y = reduce_list(x)
[output] >> ['A','B','A','B']

OR

>> x = df[df['instance']=='group_one']['value'].values # ['A','A','A','B','B','B']
>> y = reduce_list(x)
[output] >> ['A','B']

I've tried a few approaches with collections and dictionaries, but I can't wrap my head around getting farther than the following (unrelated to collections attempts):

for group in df['instance'].unique():
    val_arr = df[df['instance'] == group]['value'].values
    unique_vals = np.unique(val_arr)
    ...<then what to do?>

since dictionaries need unique keys and I may need to dynamically create the keys (e.g., A_1, B_1, A_2), but then I also need to keep in mind preserving the order.

I feel like I'm overlooking something obvious. So any help is greatly appreciated!

pandas dataframe python
2 Answers

Use itertools.groupby

from itertools import groupby
reduced = [k for k, _ in groupby(df['value'])]
print(reduced)

Output

['A', 'B', 'A', 'B', 'A', 'B']

If you needed by each group of instance, group first, then apply to each instance group:

res = [[k for k, _ in groupby(vs)] for k, vs in df.groupby('instance')['value']]
print(res)

Output

[['A', 'B', 'A', 'B'], ['A', 'B']]
user_40015920

It works for the lists, i might hand mis understood the context.

def reduce_list(x):
    unique = []
    reduced = []
    for k in x:
        if k not in unique:
            unique.append(k)
    # now we have the uniques.
    for k in range(len(x)-1):
        if x[k] != x[k+1]:
            reduced.append(x[k])
    if x[len(x)-1] != reduced[len(reduced)-1]:
        reduced.append(x[len(x)-1])
    return reduced

This is a loop intensive implementation of the code.

First loop collects the uniques which is very easy to understand.

The second loop, checks if two consecutive elements are different. If they are, it appends the one at the previous position to the loop. However, this loop fails when you have repetitive ending.

Therefore, you have to add an additional check, which sees if the last element at x is similar or different from the last element of reduced if not, it appends it.

user_130181130
Related