Creating a list of sentences from a file and adding it into a dataframe

Admin 1
Viewed 416

I am using the code below to create a list of sentences from a file document. The function will return a list of sentences.


def extract_sentences(file):
    content = nlp(file)
    sentences = list(content.sents)
    return sentences

After that, I want to add each sentence in a dataframe, under the column "sentence". The problem is that in the dataframe, the sentences appear like a list of words, divided by comma, eg: (this, process, includes, different, stages... ). But I want it to appear like: this process includes different stages

pandas spacy dataframe nlp python
2 Answers

content.sents is a generator object that holds spacy.tokens.span.Span objects.

If you want to have a list of strings as output, you can use

def extract_sentences(file):
    content = nlp(file)
    return [x.text for x in content.sents]

Note the .text property returns the textual representation of the span object.

user_38329700

sentences is a list per your function. You may want to change your return statement to return a string instead. The full function would therefore look like:

def extract_sentences(file):
    content = nlp(file)
    sentences = list(content.sents)
    return " ".join(x.text for x in sentences)
user_98576310
Related