Align words to form bracketed string

user_9775417 0
Viewed 36

Assume, I have the following bracketed string with punctuation:

s = "(S (NP-SBJ (NP (NP (NNP Ambassador) (NNP Paul) (NNP Nitze) (POS 's)) (NN statement)) (PRN (-LRB- -LRB-) (NP (NP-TTL (NNP Notable) (CC &) (NNP Quotable)) (, ,) (NP-TMP (NNP Sept.) (CD 20))) (-RRB- -RRB-) (, ,) (`` ``)) (S (SBAR-ADV (IN If) (S (NP-SBJ (PRP you)) (VP (VBP have) (NP (NP (DT a) (CD million) (NNS people)) (VP (VBG working) (PP (IN for) (NP (PRP you)))))))) (, ,) (NP-SBJ (NP (DT every) (JJ bad) (NN thing)) (SBAR (WHNP-1 (WDT that)) (S (VP (VBZ has) (NP (NP (CD one) (NN chance)) (PP (IN in) (NP (DT a) (CD million))) (PP (IN of) (S-NOM (VP (VBG going) (ADVP-CLR (NN wrong)))))))))) (VP (MD will) (VP (VB go) (ADVP-CLR (JJ wrong)) (ADVP-TMP (ADVP (IN at) (JJS least)) (IN once) (NP-ADV (DT a) (NN year)))))) (, ,) ('' '')) (VP (VBZ is) (NP-PRD (NP (DT a) (ADJP (RB pretty) (JJ negative)) (NN way)) (PP (IN of) (S-NOM (VP (VBG looking) (PP-CLR (IN at) (NP (NNS things)))))))) (. .))"

And, a reference list of punctutations I need to remove:

punctuation_words = ['.', ',', ':', '-LRB-', '-RRB-', '\'\'', '``', '--', ';',
                     '-', '?', '!', '...', '-LCB-', '-RCB-']
currency_tags_words = ['#', '$', 'C$', 'A$', 'US$']
filterwords = punctuation_words + currency_tags_words

I want to get an output like:

out = "(S (NP-SBJ (NP (NP (NNP Ambassador) (NNP Paul) (NNP Nitze) (POS 's)) (NN statement)) (PRN (NP (NP-TTL (NNP Notable) (CC &) (NNP Quotable)) (NP-TMP (NNP Sept.) (CD 20)))) (S (SBAR-ADV (IN If) (S (NP-SBJ (PRP you)) (VP (VBP have) (NP (NP (DT a) (CD million) (NNS people)) (VP (VBG working) (PP (IN for) (NP (PRP you)))))))) (NP-SBJ (NP (DT every) (JJ bad) (NN thing)) (SBAR (WHNP-1 (WDT that)) (S (VP (VBZ has) (NP (NP (CD one) (NN chance)) (PP (IN in) (NP (DT a) (CD million))) (PP (IN of) (S-NOM (VP (VBG going) (ADVP-CLR (NN wrong)))))))))) (VP (MD will) (VP (VB go) (ADVP-CLR (JJ wrong)) (ADVP-TMP (ADVP (IN at) (JJS least)) (IN once) (NP-ADV (DT a) (NN year))))))) (VP (VBZ is) (NP-PRD (NP (DT a) (ADJP (RB pretty) (JJ negative)) (NN way)) (PP (IN of) (S-NOM (VP (VBG looking) (PP-CLR (IN at) (NP (NNS things)))))))))"

Here's what I tried so far:

import nltk

t = nltk.Tree.fromstring(s)
sent = " ".join(item[0] for item in t.pos())
sent_without_punct = " ".join([item for item in sent.split() if item not in filterwords])
print(sent_without_punct)
# "Ambassador Paul Nitze 's statement Notable & Quotable Sept. 20 If you have a million people working for you every bad thing that has one chance in a million of going wrong will go wrong at least once a year is a pretty negative way of looking at things"

This gives me the correct output without punctuation. But I am having difficulty in merging it back to get the bracketed string similar to out.

Edit: The POS tag isn't relevant here. So, we can replace it with a start symbol, "S", if it helps, like so:

"(S (S (S (S (S Ambassador) (S Paul) (S Nitze) (S 's)) (S statement)) (S (S -LRB-) (S (S (S Notable) (S &) (S Quotable)) (S ,) .... "
nltk python regex string
1 Answers

You want to remove patterns like (A - A), where the text beside ( and its matching ) is identical and comes from your filter strings.

You can use

import re
punctuation_words = ['.', ',', ':', '-LRB-', '-RRB-', '\'\'', '``', '--', ';',
                     '-', '?', '!', '...', '-LCB-', '-RCB-']
currency_tags_words = ['#', '$', 'C$', 'A$', 'US$']
filterwords = punctuation_words + currency_tags_words
filter_rx = "|".join(sorted(map(re.escape,filterwords), key=len, reverse=True))
rx = r"\s*\(({0}) \1\)".format(filter_rx)
text = "(S (NP-SBJ (NP (NP (NNP Ambassador) (NNP Paul) (NNP Nitze) (POS 's)) (NN statement)) (PRN (-LRB- -LRB-) (NP (NP-TTL (NNP Notable) (CC &) (NNP Quotable)) (, ,) (NP-TMP (NNP Sept.) (CD 20))) (-RRB- -RRB-) (, ,) (`` ``)) (S (SBAR-ADV (IN If) (S (NP-SBJ (PRP you)) (VP (VBP have) (NP (NP (DT a) (CD million) (NNS people)) (VP (VBG working) (PP (IN for) (NP (PRP you)))))))) (, ,) (NP-SBJ (NP (DT every) (JJ bad) (NN thing)) (SBAR (WHNP-1 (WDT that)) (S (VP (VBZ has) (NP (NP (CD one) (NN chance)) (PP (IN in) (NP (DT a) (CD million))) (PP (IN of) (S-NOM (VP (VBG going) (ADVP-CLR (NN wrong)))))))))) (VP (MD will) (VP (VB go) (ADVP-CLR (JJ wrong)) (ADVP-TMP (ADVP (IN at) (JJS least)) (IN once) (NP-ADV (DT a) (NN year)))))) (, ,) ('' '')) (VP (VBZ is) (NP-PRD (NP (DT a) (ADJP (RB pretty) (JJ negative)) (NN way)) (PP (IN of) (S-NOM (VP (VBG looking) (PP-CLR (IN at) (NP (NNS things)))))))) (. .))"
print( re.sub(rx, "", text) )

See the Python demo and the regex demo.

The regex is of the \s*\(<PATTERN> \1\) type, and it matches

  • \s* - zero or more whitespaces
  • \( - a ( char
  • (\-LRB\-|\-RRB\-|\-LCB\-|\-RCB\-|\.\.\.|\-\-|US\$|C\$|A\$|\.|''|``|\-|\?|\#|\$|,|:|;|!) - one of the filters
  • - space
  • \1 - Group 1 backreference matching the same text as captured in Group 1
  • \) - a ) char.
user_38329700
Related