Context

This is a case of approximate string matching or fuzzy matching. There is good materials and libraries about that.

There are different libraries and ways to cover this. I will restrict to relatively simple libraries

Some cool libraries:

from fuzzywuzzy import process
import pandas as pd
import string

First Part

Let's place the data to play around. I try to reproduce the above example, hope it is fine.

# Set up dataframe
d = {'originals': [["Water","PEG-60 Hydrogenated Castor Oil"],
                   ["PEG-60 Hydrnated Castor Oil"],
                   ["wter"," PEG-60 Hydrnated Castor Oil"],
                   ['Vitamin E']],
     'correct': [["Water","PEG-60 Hydrogenated Castor Oil"],
                 ["PEG-60 Hydrogenated Castor Oil"],
                 ['Water', 'PEG-60 Hydrogenated Castor Oil'],
                 ['Tocopherol (Vitamin E)']]}
df = pd.DataFrame(data=d)
print(df)
                                 originals                                  correct
0  [Water, PEG-60 Hydrogenated Castor Oil]  [Water, PEG-60 Hydrogenated Castor Oil]
1            [PEG-60 Hydrnated Castor Oil]         [PEG-60 Hydrogenated Castor Oil]
2     [wter,  PEG-60 Hydrnated Castor Oil]  [Water, PEG-60 Hydrogenated Castor Oil]
3                              [Vitamin E]                 [Tocopherol (Vitamin E)]

From the above we have the statement of the problem: we have some original wording and want to change it.

Which are the correct options for us:

strOptions = ['Water', "Tocopherol (Vitamin E)",
             "Vitamin D", "PEG-60 Hydrogenated Castor Oil"]

This functions are going to help us. I try to document them well.

def function_proximity(str2Match,strOptions):
    """
    This function help to get the first guess by similiarity.

    paramters
    ---------
    str2Match: string. The string to match.
    strOptions: list of strings. Those are the possibilities to match.
    """
    highest = process.extractOne(str2Match,strOptions)
    return highest[0]
def check_strings(x, strOptions):
    """
    Takes a list of string and give you a list of string best matched.
    :param x: list of string to link / matched
    :param strOptions:
    :return: list of string matched
    """
    list_results = []
    for i in x:
        i=str(i)
        list_results.append(function_proximity(i,strOptions))
    return list_results

Let's apply to the dataframe:

df['solutions_1'] = df['originals'].apply(lambda x: check_strings(x, strOptions))

Let's check the results, by comparing the columns.

print(df['solutions_1'] == df['correct'])
0    True
1    True
2    True
3    True
dtype: bool

As you can see the solution work in the four cases.

Second Part

Problem example solution: You have Water Vtamin D should become Water, Vitamin D.

Let's create a list of valid words.

list_words = []
for i in strOptions:
    print(i.split(' '))
    list_words = list_words + i.split(' ')
# Lower case and remove punctionation
list_valid_words = []
for i in list_words:
    i = i.lower()
    list_valid_words.append(i.translate(str.maketrans('', '', string.punctuation)))
print(list_valid_words)
['water', 'tocopherol', 'vitamin', 'e', 'vitamin', 'd', 'peg60', 'hydrogenated', 'castor', 'oil']

That if the list of words that are valid.

def remove_puntuation_split(x):
    """
    This function remove puntuation and split the string into tokens.
    :param x: string
    :return: list of proper tokens
    """
    x = x.lower()
    # Remove all puntuation
    x = x.translate(str.maketrans('', '', string.punctuation))
    return x.split(' ')

tokens = remove_puntuation_split(x)
# Clean tokens
clean_tokens = [function_proximity(x,list_valid_words) for x in tokens]
# Matched tokens with proper selection
tokens_clasified = [function_proximity(x,strOptions) for x in tokens]
# Removed repeated
tokens_clasified =  list(set(tokens_clasified))
print(tokens_clasified)
['Vitamin D', 'Water']

That is as required initially. However those could fail a bit, particularly when Vitamin E and D are combined.

References

• https://www.datacamp.com/community/tutorials/fuzzy-string-python
• https://pbpython.com/record-linking.html
• https://pbpython.com/record-linking.html

This answer builds upon @Rafaels answer.

1. process.extractOne in FuzzyWuzzy uses the scorer fuzz.WRatio by default. This is a combination of multiple scorers provided by FuzzyWuzzy, that works well for the dataset Seatgeek is working with. So you might want to try around with other scorers to see which one performs best for you. However note, that quite a few of your elements might be hard to distinguish using the edit distance. E.g. Vitamin E <-> Vitamin D only need a single edit, even though they are something completely different. The same behavior also occurs with glycereth-7

2. FuzzyWuzzy is relatively slow, so when your working with a bigger dataset you might want to use RapidFuzz instead (I am the author) which provides similar algorithms, but has a better performance.

3. process.extractOne preprocesses the input strings by default (lowercases and replaces non alphanumeric characters with whitespaces). Since your probably searching for elements multiple times it would make sense to preprocess the possible choices once ahead of time and deactivate this behavior to safe some time:

process.extractOne(str2Match,strOptions, processor=None)

Differences between RapidFuzz and FuzzyWuzzy

Since you reported differences in the results between RapidFuzz and FuzzyWuzzy here are some possible reasons:

1. I do not round results. So you wil get a floating point like 42.22 instead of 42 as a result
2. In case your not using the fast FuzzyWuzzy implementation, that uses python-Levenshtein you might get different results, since it uses difflib which is a different metric. It produces very similar results most of the time but not always
3. In case your using the fast implementation any partial ratio like partial_ratio, WRatio ... might return wrong results in FuzzyWuzzy since partial_ratio is broken (see here)
4. Passing processor=None to extract/extractOne has a different meaning in RapidFuzz and FuzzyWuzzy. In RapidFuzz it will deactivate preprocessing, while in FuzzyWuzzy it will still use the default of the score. As an example for

extract(..., scorer=fuzz.WRatio, processor=None)

FuzzyWuzzy will still preprocess the strings inside WRatio, so there is no way to deactivate preprocessing. I personally think this is a bad design, so I changed it to give the user the possibility to deactivate the processor, which is likely what you want to achieve when passing processor=None

I expanded on the other answers to make it work on the provided list. It's sort of an algorithm using fuzzywuzzy and seems to work for a case like vitamin e.

def merge_scores(text, matches, match_func):
    new_scores = []
    for match in matches:
        new_scores.append((match[0], (match[1] + match_func(match[0], text)) / 2))
    return sorted(new_scores, key=lambda m:m[1], reverse=True)

def get_best_match(text):
    fuzz_matches = process.extractBests(text, INGREDIENTS, limit=10, scorer=fuzz.ratio)
    if fuzz_matches[0][1] < 80 or fuzz_matches[0][1] == fuzz_matches[1][1]:
        fuzz_matches = process.extractBests(text, INGREDIENTS, limit=10, scorer=fuzz.token_set_ratio)
        # Combine only if the top 5 aren't perfect matches
        if fuzz_matches[4][1] != 100:
            fuzz_matches = merge_scores(text, fuzz_matches, fuzz.ratio)
    if fuzz_matches[0][1] == fuzz_matches[1][1]:
        fuzz_matches = process.extractBests(text, INGREDIENTS, limit=10, scorer=fuzz.WRatio)
    if fuzz_matches[0][1] == fuzz_matches[1][1]:
        return '', 0
    return fuzz_matches[0]

I copy and paste the list from https://pastebin.com/QXabQXWP in file list_terms_v3.txt.

"""

References
https://github.com/seatgeek/fuzzywuzzy
https://www.datacamp.com/community/tutorials/fuzzy-string-python

Questions Stackoverflow: https://stackoverflow.com/questions/65844582/query-segmentation-with-spell-check

"""

from fuzzywuzzy import process
from fuzzywuzzy import fuzz
import pandas as pd
import string

#strOptions = ['Water', "Tocopherol (Vitamin E)",
#             "Vitamin D", "PEG-60 Hydrogenated Castor Oil"]
strOptions = pd.read_csv("list_terms_v3.txt", sep='\n', header=None)
strOptions = strOptions[0].tolist()
print(strOptions.__len__())

"""
Create a list of key workds
"""
list_words = []
for i in strOptions:
    print(i.split(' '))
    list_words = list_words + i.split(' ')
# Lower case and remove punctionation
list_valid_words = []
for i in list_words:
    i = i.lower()
    list_valid_words.append(i.translate(str.maketrans('', '', string.punctuation)))


def function_proximity(str2Match,strOptions = strOptions):
    """
    This function help to get the first guess by similiarity.

    paramters
    ---------
    str2Match: string. The string to match.
    strOptions: list of strings. Those are the possibilities to match.
    """
    highest = process.extractOne(str2Match,strOptions)
    return highest[0]
def check_strings(x, strOptions):
    """
    Takes a list of string and give you a list of string best matched.
    :param x: list of string to link / matched
    :param strOptions:
    :return: list of string matched
    """
    list_results = []
    for i in x:
        i=str(i)
        list_results.append(function_proximity(i))
    return list_results

"""
2 Option
"""


def remove_puntuation_split(x):
    """
    This function remove puntuation and split the string into tokens.
    :param x: string
    :return: list of proper tokens
    """
    x = x.lower()
    # Remove all puntuation
    x = x.translate(str.maketrans('', '', string.punctuation))
    return x.split(' ')

def clean_tokens_funct(x, strOptions=strOptions):
    if x in strOptions:
        return x
    else:
        return function_proximity(x,strOptions=strOptions)


def searcher(x):
    tokens = remove_puntuation_split(x)
    clean_tokens = [clean_tokens_funct(x, strOptions=list_valid_words) for x in tokens]
    # Matched tokens with proper selection
    tokens_clasified = [function_proximity(x,strOptions=strOptions) for x in tokens]
    # Removed repeated
    tokens_clasified =  list(set(tokens_clasified))
    return tokens_clasified

Up to here everything fine, to check:

x = "Water Vtamin E"
print(searcher(x))
['Tocopheryl Acetate (Vitamin E)', 'Water']

However previous could file in some cases. The problem is that Vitamin D does not appear in the list, but if added everything should be fine.

x = "Water Vtamin D"
print(searcher(x))
['Tocopheryl Acetate (Vitamin E)', 'Ceramide', 'Water']

Or another case for problem:

x = "Water  D"
print(searcher(x))
['Ceramide', 'Water']

In case you just want to compare with the list you have then use function_proximity

x = 'Actinidia Chinensis Fruit'
print(function_proximity(x))
Luffa Cylindrica Fruit/Leaf/Stem Extract


x = 'Nelumbo Nucifera Callus Culture'
print(function_proximity(x))
Nelumbo Nucifera Callus Culture Extract