Select relevant strings from a list of sentence tokens without changing their order

user_16965610 0
Viewed 93

I'm looking to select relevant sentence tokens from a text. The selection has to be made if strings in the list 'keywords' are present and strings in the list 'avoid' are not present in the sentence tokens. However, the sentence tokens containing strings in the list 'avoid' have to be selected if strings in the list 'essentials' are present.

Here's the code I tried running:

#sentences is the list containing sentence tokens

facts = []

for sent in sentences:
    for keyword in keywords:
        if keyword in sent:
            facts.append(sent)

for fact in facts[:]:
    for i in avoid[:]:
        for essential in essentials[:]:
            if i in fact and essential not in fact:
                facts.remove(fact)

This however does not include the sentence tokens of the second category i.e. the ones with essential strings along with the ones to be avoided.

What am I doing wrong and is there a better way to do this? Thanks in advance

I tried another way but it messes up the order of the sentence tokens. It goes like this:

for sent in sentences:
    for keyword in keywords:
        if keyword in sent:
            facts.append(sent)

relevant_facts = [fact for fact in facts if not (i in fact for i in avoid)]

for sent in facts:
    for essential in essentials:
        if essential in sent:
            relevant_facts.append(sent)
tokenize nltk nlp python
1 Answers

so I went with another way to do it.

It might not be optimal in terms of complexity, but it is easier to understand. So I created 3 functions returning if the sentence has a keyword, an avoid or an essential.

Then for each sentence, if it has an Essential, we add it, if it has a keyword but no avoid, we add it too. Otherwise we just skip it.

Here is the code :

facts = []

def hasKeyword(sentence):
  for keyword in keywords:
    if keyword in sentence:
      return True
  return False

def hasAvoid(sentence):
  for avoid in avoids:
    if avoid in sentence:
      return True
  return False

def hasEssential(sentence):
  for essential in essentials:
    if essential in sentence:
      return True
  return False


for sent in sentences:
  if hasEssential(sent):
    facts.append(sent)
  elif hasKeyword(sent) and not hasAvoid(sent):
    facts.append(sent)
user_146508800
Related