TypeError: Argument 'other' has incorrect type (expected spacy.tokens.token.Token, got str)

user_17169994 0
Viewed 72

I'm looking to get all sentences in a text file that contain at least one of the conjunctions in the list "conjunctions". However, when applying this function for the text in the variable "text_to_look" like this:

import spacy
lang_model = spacy.load("en_core_web_sm")
text_to_look = "A woman is looking at books in a library. She's looking to buy one, but she hasn't got any money. She really wanted to book, so she asks another customer to lend her money. The man accepts. They get along really well, so they both exchange phone numbers and go their separate ways."

def get_coordinate_sents(file_to_examine):
    conjunctions = ['and', 'but', 'for', 'nor', 'or', 'yet', 'so']
    text = lang_model(file_to_examine)
    sentences = text.sents
    for sentence in sentences:
        coord_sents = []
        if any(conjunction in sentence for conjunction in conjunctions):
            coord_sents.append(sentence)
    return coord_sents
        
wanted_sents = get_coordinate_sents(text_to_look)

I get this error message :

TypeError: Argument 'other' has incorrect type (expected spacy.tokens.token.Token, got str)

There seems to be something about spaCy that I'm not aware of and prevents me from doing this...

spacy python
1 Answers

While the problem lies in the fact that conjunction is a string and sentence is a Span object, and to check if the sentence text contains a conjunction you need to access the Span text property, you also re-initialize the coord_sents in the loop, effectively saving only the last sentence in the variable. Note a list comprehension looks preferable in such cases.

So, a quick fix for your case is

def get_coordinate_sents(file_to_examine):
    conjunctions = ['and', 'but', 'for', 'nor', 'or', 'yet', 'so']
    text = lang_model(file_to_examine)
    return [sentence for sentence in text.sents if any(conjunction in sentence.text for conjunction in conjunctions)]

Here is my test:

import spacy
lang_model = spacy.load("en_core_web_sm")

text_to_look = "A woman is looking at books in a library. She's looking to buy one, but she hasn't got any money. She really wanted to book, so she asks another customer to lend her money. The man accepts. They get along really well, so they both exchange phone numbers and go their separate ways."
file_to_examine = text_to_look
conjunctions = ['and', 'but', 'for', 'nor', 'or', 'yet', 'so']

text = lang_model(file_to_examine)
sentences = text.sents
coord_sents = [sentence for sentence in sentences if any(conjunction in sentence.text for conjunction in conjunctions)]

Output:

>>> coord_sents
[She's looking to buy one, but she hasn't got any money., She really wanted to book, so she asks another customer to lend her money., They get along really well, so they both exchange phone numbers and go their separate ways.]

However, the in operation will find nor in north, so in crimson, etc.

You need a regex here:

import re
conjunctions = ['and', 'but', 'for', 'nor', 'or', 'yet', 'so'] 
rx = re.compile(fr'\b(?:{"|".join(conjunctions)})\b')

def get_coordinate_sents(file_to_examine):
    text = lang_model(file_to_examine)
    return [sentence for sentence in text.sents if rx.search(sentence.text)]
user_38329700
Related