Error with count tags (nltk) in column dataframe

Viewed 61
# Extracting definition from different words in each sentence
# Extractinf from ecah row the, NOUN, VERBS, NOUN Plural 
text = data['Omschrijving_Skill_without_stopwords'].tolist()
tagged_texts = pos_tag_sents(map(word_tokenize, text))
data['pos'] = pos_tag_sents( data['Omschrijving_Skill_without_stopwords'].apply(word_tokenize).tolist() )
data['Nouns'] = data['pos'].apply(lambda row: [i[0] for i in row if i[1] == 'NN'])
data['Verbs'] = data['pos'].apply(lambda row: [i[0] for i in row if i[1] == 'VB'])
data['Adj'] = data['pos'].apply(lambda row: [i[0] for i in row if i[1] == 'JJ'])
data['NOUN Pltagged_textsural'] = data['pos'].apply(lambda row: [i[0] for i in row if i[1] == 'NNS'])
data['Omschrijving_Skill_without_stopwords'] = data['Omschrijving_Skill_without_stopwords'].astype(str)
data['Omschrijving_Skill_without_stopwords'] = data['Omschrijving_Skill_without_stopwords'].str.split("t", n =1, expand=True)

the output is

0       [(beschikbaarheid, NN), (mili, NN)]
1                               [(par, NN)]
2                             [(opera, NN)]
3                              [(mili, NN)]
4                             [(opera, NN)]
                       ...                 
4696                             [(on, IN)]
4697                 [(veiligheidssys, NN)]
4698                                     []
4699                              [(a, DT)]
4700                              [(a, DT)]
Name: pos, Length: 4701, dtype: object

I want to count from data['pos'] how many nouns, verbs etc I have so I want to have an output:

('NN':3, 'DT':2 )

I tried the following code:

Counter([j for i, j in pos_tag(word_tokenize(tagged_text))])

but I got an error:

TypeError: 'function' object is not iterable

I have the idea that because I have an list, I can't use counter. I tried to convert the data['Omschrijving_skill_without_stopwords'] to a dictionary But that does not work, can any one help me to count my tags so that I know many tags I have?

Thank you

1 Answers

I change the tagged_texts, i think i don't understand how i have to change it @nikerosbut this is the line of code.

tagged_text = itertools.chain.from_iterable(tagged_texts)
tagged_texts = pos_tag_sents(map(word_tokenize, tagged_text))

And then ik tried to use this line of code erorr:

expected string or bytes-like object
Related