Remove any duplicates in a list of lists when order doesn't matter (eg. [1,2,3] and [1,3,2] are duplicate sets)?

user_4967582 0
Viewed 296

I've been deleting all occurrences unintentionally. I would like to keep at least one set of occurrences.

For example, I have [[1,2,3],[1,3,2],[4,5,6],[5,6,4]] and the desired output would be akin to [[1,2,3],[4,5,6]].

s = 1,2,3,4,5,6
c = [[1,2,3],[4,5,6],[4,6,5]]

remove_sets = []
for a in range(0, len(c)):
    for b in permutations(c[a], 3):
        # my idea is that if list(b) != c[a]
        # it should not delete all occurences.
        if list(b) != c[a]:
            if list(b) in c:
                remove_sets.append(list(b))

# delete those occurences.
for cc in range(0, len(remove_sets)):
    if remove_sets[cc] in c:
        del c[c.index(remove_sets[cc])]

Unintended Result/Output

[[1,2,3]]

My desired output would be

[[1,2,3],[4,5,6]]

Question

Is there a function for removing these duplicate sets where order is switched around?

duplicates set python
1 Answers

groupby works if your duplicate sublists are already adjacent and you need to squeeze them into single units:

>>> from itertools import groupby
>>> [next(v) for _, v in groupby(c, sorted)]
[[1, 2, 3], [4, 5, 6]]

Caling sorted ignores order when grouping, so we can skim the first item from each group to obtain your result.

Otherwise, for the general case, using a dictionary comprehension like

>>> list({tuple(sorted(x)): x for x in c}.values())
[(1, 2, 3), (4, 6, 5)]

works but it only selects the last-seen item in c. If you reverse-iterate c, you'll get the first-seen:

>>> list({tuple(sorted(x)): x for x in c[::-1]}.values())[::-1]
[(1, 2, 3), (4, 5, 6)]

Be aware that sorted is O(n(log(n))), resulting in an overall complexity of

longest_len = max(map(len, c))
O(n * longest_len * log(longest_len))

If you need to scale to large inner lists, consider collections.Counter instead of sorted.

user_62433520
Related