Get the list of tokens with BI tags with compound words

Viewed 13

I have two Python lists. One is the tokens list and the other is BIO tagged list.

tokens_list=['I', 'think', 'it', "'s", 'harder', 'to', 'have', 'the', 'memory', 'care', 'area', 'of', 'the', 'community', 'clean', '.', 'It', 'often', 'does', 'not', 'have', 'a', 'fresh', 'smell', ',', 'which', 'I', 'understand', '.'] 

The BIO tagged list contains 1's and 2's.

BI_list=[0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0]

1 corresponds to the B tag while the 2 corresponds to I tag. What I want is to get a list corresponding to BI tags with tokens combined if '2' follows '1' in BI_list.

My attempt:

tokens_list=['I', 'think', 'it', "'s", 'harder', 'to', 'have', 'the', 'memory', 'care', 'area', 'of', 'the', 'community', 'clean', '.', 'It', 'often', 'does', 'not', 'have', 'a', 'fresh', 'smell', ',', 'which', 'I', 'understand', '.']

BI_list=[0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 2, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0]

Output_BI_words=[]
tags=[]
for each,word in enumerate(tokens_list):
  if (BI_list[each]==1 or BI_list[each]==2):
    tags.append(BI_list[each])
    Output_BI_words.append(word)

print(Output_BI_words)
print(tags)

Output is like:

['memory', 'care', 'area', 'community', 'smell']

[1, 2, 2, 1, 1]

The output should be like this: Output_list=['memory care area', 'community', 'smell']

0 Answers
Related