Python: Combine results of multiple pipelines

Question

So, I am new to Python and I have been working on Stanford NLP. So, below are the two pipelines that I created.

nlp = stanza.Pipeline(lang='en', processors='tokenize,mwt,pos,lemma,depparse')
doc = nlp('This is Prince Georges medic 829 were en route with a fourteen year old male, gunshot wound.  The patient has two wounds both, um, both approximately the same size.  )
for i, sent in enumerate(doc.sentences):
  print(f'====== Sentence {i+1} tokens =======')
  print(*[f'word: {word.text}	upos: {word.upos}	xpos: {word.xpos}	lemma: {word.lemma}	deprel: {word.deprel}' for word in sent.words], sep='
')

Sample Output Pipleine1:

word: fourteen  upos: NUM   xpos: CD    lemma: fourteen deprel: nummod

word: year  upos: NOUN  xpos: NN    lemma: year deprel: obl:npmod

word: old   upos: ADJ   xpos: JJ    lemma: old  deprel: amod

word: male  upos: NOUN  xpos: NN    lemma: male deprel: nmod

nlp1 = stanza.Pipeline(lang='en', processors='tokenize,ner')
doc1 = nlp1("This is Prince Georges medic 829 were en route with a fourteen year old male, gunshot wound.  The patient has two wounds both, um, both approximately the same size.)
#doc1 = nlp1("Prior to you with a 14 year old male, who got out of bed and had some sort of syncopal episode, and took a fall striking his head.")
print(*[f'token: {token.text}	ner: {token.ner}' for sent in doc1.sentences for token in sent.tokens], sep='
')

Sample Output Pipeline2:

token: fourteen ner: B-DATE

token: year     ner: I-DATE

token: old      ner: E-DATE

token: male     ner: O

The first pipeline tags features like Part-of-Speech, does lemmatization, and dependency parsing (dependency relations between words). The second pipeline is for named entity recognition that returns the entity type.

I know how to dump the output of the two pipelines individually into a text/json format. What I want, is to combine the outputs of the two pipeline and then dump it into a text/json format. Is there a way that I can combine the outputs of multiple pipelines?

For example, if I combine, xpos, deprel, token and ner for "fourteen, year and old" it should return age as fourteen. Thanks for help.

user_1712292 · Answer

The way I would recommend doing this is by changing the call to the second pipeline to use the document object that is returned from the first (you're doc variable).

the second code block would then be this instead:

nlp1 = stanza.Pipeline(lang='en', processors='tokenize,ner')
doc = nlp1(doc)

You can also pass in more than one doc at a time now as of version 1.2: https://stanfordnlp.github.io/stanza/pipeline.html#processing-multiple-documents

Also, if you have an arbitrary list of pipelines you want to apply in succession to a piece of text, you can do this:

doc = None
for pipeline in pipelines:
    if doc:
        doc = pipeline(doc)
    else:
        doc = pipeline(text)

Python: Combine results of multiple pipelines

1 Answers