Find all words in a sentence related to a keyword

user_12645703 0
Viewed 340

I have the following text and want to isolate a part of the sentence related to a keyword, in this case keywords = ['pizza', 'chips'].

text = "The pizza is great but the chips aren't the best"

Expected Output:

{'pizza': 'The pizza is great'}
{'chips': "the chips aren't the best"}

I have tried using the Spacy Dependency Matcher but admittedly I'm not quite sure how it works. I tried the following pattern for chips which yields no matches.

import spacy
from spacy.matcher import DependencyMatcher

nlp = spacy.load("en_core_web_sm")

pattern = [
  {
    "RIGHT_ID": "chips_id",
    "RIGHT_ATTRS": {"ORTH": "chips"}
  },
    
  {
    "LEFT_ID": "chips_id",
    "REL_OP": "<<",
    "RIGHT_ID": "other_words",
    "RIGHT_ATTRS": {"POS": '*'}
  }
]

matcher = DependencyMatcher(nlp.vocab)
matcher.add("chips", [pattern])

doc = nlp("The pizza is great but the chips aren't the best")
for id_, (_, other_words) in matcher(doc):
    print(doc[other_words])

Edit:

Additional example sentences:

example_sentences = [
    "The pizza's are just OK, the chips is stiff and the service mediocre",
    "Then the mains came and the pizza - these we're really average - chips had loads of oil and was poor",
    "Nice pizza freshly made to order food is priced well, but chips are not so keenly priced.",
    "The pizzas and chips taste really good and the Tango Ice Blast was refreshing"
]
spacy nltk nlp python
2 Answers

you could use the following function :

def spliter(text : str , keyword :list, number_of_words:int):
    L = text.split()
    sentences = dict()
    for k in L :
        if k in keyword :
            n = L.index(k)
            if len(L) -n -1 > number_of_words :
                sentences.update({k:' '.join(L[n : n + number_of_words])})
            else :
                sentences.update({k:' '.join(L[n :])})
    return sentences

Note : number_of_word define how many word you want to get after the desired keyword

Output : for number_of_words = 3 you get :

{'pizza': 'pizza is great', 'chips': "chips aren't the best"}
user_159197830

Here is my attempt at a very limited solution to your problem, since I do not know how extensive you will want this to be.

I utilized code from this answer in order to address the problem.

import spacy
import re

en = spacy.load('en_core_web_sm')

text = "The pizza is great but the chips aren't the best"

doc = en(text)

seen = set() # keep track of covered words

chunks = []
for sent in doc.sents:
    heads = [cc for cc in sent.root.children if cc.dep_ == 'conj']

    for head in heads:
        words = [ww for ww in head.subtree]
        for word in words:
            seen.add(word)
        chunk = (' '.join([ww.text for ww in words]))
        chunks.append( (head.i, chunk) )

    unseen = [ww for ww in sent if ww not in seen]
    chunk = ' '.join([ww.text for ww in unseen])
    chunks.append( (sent.root.i, chunk) )

chunks = sorted(chunks, key=lambda x: x[0])


output_dict = {}

for np in doc.noun_chunks:
    insensitive_the = re.compile(re.escape('the '), re.IGNORECASE)
    new_np = insensitive_the.sub('',np.text)
    output_dict[new_np]=''

for ii, chunk in chunks:
    #print(ii, chunk)
    for key in output_dict:
        if key in chunk:
            output_dict[key]=chunk

print(output_dict)

The output I get is: enter image description here

I am aware there are a few problems:

  1. The conjunction 'but' should not be in the value of the pizza key.
  2. The word "are n't" should be aren't in the second value of the dictionary.

However, I believe we can fix this if we know more information about what sort of sentences you are dealing with. For instance, we might have a list of conjunctions that we can strip from all the values of the dict if the sentences are simple enough.

Update with example sentences: enter image description here

As you can see, I think SpaCy struggles a bit with the punctuation, as well as knowing that you only want food items as nouns in the dictionary, presumably.

user_109193490
Related