How to apply a sentiment classifier to a dataframe

user_12020223 0
Viewed 1049

I have a dataframe which contains survey answers. Three of those columns are open-ended answers. Using HuggingFace NLP I'm using a pre-trained sentiment analysis classifier. Please find the code below:

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline
model_name = "nlptown/bert-base-multilingual-uncased-sentiment"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
classifier = pipeline('sentiment-analysis', model=model, tokenizer=tokenizer)
classifier("This community is so helpful!")

The results for the classifier test is: "[{'label': '5 stars', 'score': 0.800311}]

What I'd like to do is have the classifier run on my open-ended responses and, in new columns in my dataframe, have it include the stars and ranking score.

Any help would be greatly appreciated.

edit: I uploaded the dataset through a local csv. The dataframe column name I want to work with is "Q72"

huggingface-transformers pandas python
1 Answers

Apply model on a column and create another column using assign function:

df = (
    df
    .assign(sentiment = lambda x: x['Q72'].apply(lambda s: classifier(s)))
    .assign(
         label = lambda x: x['sentiment'].apply(lambda s: (s[0]['label'])),
         score = lambda x: x['sentiment'].apply(lambda s: (s[0]['score']))
    )
)
user_39588780
Related