Find unique words in a list of lists in python

user_11332149 0
Viewed 856

I have a list of lists that I would like to iterate over using a for loop, and create a new list with only the unique words. This is similar to a question asked previously, but I could not get the solution to work for me for a list within a list

For example, the nested list is as follows:

ListofList = [['is', 'and', 'is'], ['so', 'he', 'his'], ['his', 'run']],

The desired output would be a single list:

List_Unique = [['is','and','so','he','his','run']]

I have tried the following two variations of code, but the output of all of them is a list of repeats:

unique_redundant = [] 
for i in redundant_search:
    redundant_i = [j for j in i if not i in unique_redundant]
    unique_redundant.append(redundant_i)
unique_redundant


unique_redundant = [] 
for list in redundant_search:
    for j in list:
        redundant_j = [i for i in j if not i in unique_redundant]
    unique_redundant.append(length_j)
unique_redundant

Example output given for the above two (incorrect) variations

(I ran the code on my real set of data and it gave repeating lists within lists of the same pair of words, though this isn't the actual two words, just an example):

List_Unique = [['is','and'],['is','and'],['is','and']]
for-loop list python
7 Answers

I'd suggest using the set() class union() in this way:

ListofList = [['is', 'and', 'is'], ['so', 'he', 'his'], ['his', 'run']]

set().union(*ListofList)
# => {'run', 'and', 'so', 'is', 'his', 'he'}

Explanation

It works like the following:

test_set = set().union([1])
print(test_set)
# => {1}

The asterisk operator before the list (*ListofList) unpacks the list:

lst = [[1], [2], [3]]
print(lst) #=> [[1], [2], [3]]
print(*lst) #=> [1] [2] [3]
user_52390300

First flatten the list with itertools.chain, then use set to return the unique elements and pass that into a list:

from itertools import chain

if __name__ == '__main__':
    print([{list(chain(*list_of_lists))}])
user_87483080

Use itertools.chain to flatten the list and dict.fromkeys to keep the unique values in order:

ListofList = [['is', 'and', 'is'], ['so', 'he', 'his'], ['his', 'run']]

from itertools import chain
List_Unique = [list(dict.fromkeys(chain.from_iterable(ListofList)))]
user_163434640

Just index out nested list with the help of while and acquire all the values in new list while cnt<len(listoflist)

ListofList = [['is', 'and', 'is'], ['so', 'he', 'his'], ['his', 'run']]
list_new=[]
cnt=0
while cnt<len(ListofList):
    for i in ListofList[cnt]:
        if i in list_new:
            continue
        else:
            list_new.append(i)
    cnt+=1
print(list_new)

OUTPUT

['is', 'and', 'so', 'he', 'his', 'run']
user_141411640 
flat_list = [item for sublist in ListofList for item in sublist]

# use this if order should not change
List_Unique = []
for item in flat_list:
    if item not in List_Unique:
        List_Unique.append(item)


# use this if order is not an issue
# List_Unique = list(set(flat_list))
user_99292720

You could try this:

ListofList = [['is', 'and', 'is'], ['so', 'he', 'his'], ['his', 'run']]
uniqueItems = []
for firstList in ListofList:
    for item in firstList:
        if item not in uniqueItems:
            uniqueItems.append(item)
print(uniqueItems)

It uses a nested for loop to access each item and check whether it is in uniqueItems.

user_175120960

using basic set concept, set consists of unique elements

lst = [['is', 'and', 'is'], ['so', 'he', 'his'], ['his', 'run']]
new_list = []
for x in lst:
    for y in set(x):
        new_list.append(y)
print(list(set(new_list)))


['run', 'and', 'is', 'so', 'he', 'his']
user_58580060
Related