Delete duplicates in list of dictionary depending on criteria

user_7831854 0
Viewed 51

I only found questions where people wanted to delete duplicates but not depending on a criteria which includes the value of the keys.

Lets say I have a List containing dictionaries

myList = [
  {'version': 'v1', 'updated': '2020-06-17 22:15:00+00:00'},
  {'version': 'v1', 'updated': '2020-08-24 17:37:00+00:00'},
  {'version': 'v5', 'updated': '2019-08-30 11:42:00+00:00'},
  {'version': 'v1', 'updated': '2020-03-11 22:28:00+00:00'},
  {'version': 'v7', 'updated': '2020-06-17 22:15:00+00:00'}
]

now I want to remove the duplicate dictionaries where the keys have the same value, e.g., version has the value v1 and only keep the most current one but I don't know that the value is v1 so I have to check for duplicates in version without knowing the value. After finding the duplicates I want to compare the date in update and choose the latest one and remove the duplicates due to this criteria. I can not guarantee that the latest one is the last entry, like it can be seen in myList that the second entry for v1 is the latest one, not the third one.

Resulting in

myCleanedList= [
  {'version': 'v1', 'updated': '2020-08-24 17:37:00+00:00'},
  {'version': 'v5', 'updated': '2019-08-30 11:42:00+00:00'},
  {'version': 'v7', 'updated': '2020-06-17 22:15:00+00:00'}
]
duplicates dictionary python
2 Answers

Try:

myList = [
    {"version": "v1", "updated": "2020-06-17 22:15:00+00:00"},
    {"version": "v1", "updated": "2020-08-24 17:37:00+00:00"},
    {"version": "v5", "updated": "2019-08-30 11:42:00+00:00"},
    {"version": "v1", "updated": "2020-03-11 22:28:00+00:00"},
    {"version": "v7", "updated": "2020-06-17 22:15:00+00:00"},
]

out = {}
for d in myList:
    out.setdefault(d["version"], []).append(d)

out = [
    sorted(v, key=lambda k: k["updated"], reverse=True)[0] for v in out.values()
]
print(out)

Prints:

[{'version': 'v1', 'updated': '2020-08-24 17:37:00+00:00'}, 
 {'version': 'v5', 'updated': '2019-08-30 11:42:00+00:00'}, 
 {'version': 'v7', 'updated': '2020-06-17 22:15:00+00:00'}]
user_100359850

Code with always sorted 'version' in any input order:
(input order was changed to 'version' 1-7-5 instead original 1-5-7)

myList = [
    {'version': 'v1', 'updated': '2020-06-17 22:15:00+00:00'},
    {'version': 'v7', 'updated': '2020-06-17 22:15:00+00:00'},
    {'version': 'v1', 'updated': '2020-08-24 17:37:00+00:00'},
    {'version': 'v5', 'updated': '2019-08-30 11:42:00+00:00'},
    {'version': 'v1', 'updated': '2020-03-11 22:28:00+00:00'}
]

out = {d["version"]: d for d in sorted(myList, key=lambda k: k["updated"])}
print(*sorted(out.values(), key=lambda k: k["version"]), sep='\n')

Output:

{'version': 'v1', 'updated': '2020-08-24 17:37:00+00:00'}
{'version': 'v5', 'updated': '2019-08-30 11:42:00+00:00'}
{'version': 'v7', 'updated': '2020-06-17 22:15:00+00:00'}
user_150353140
Related