Convert string to colum

user_11985685 0
Viewed 136

I have a simple data frame, and I am developing a sentiment analysis.

This is the code and the reproducible example

import transformers
from pysentimiento import SentimentAnalyzer
from pysentimiento import EmotionAnalyzer

analyzer = SentimentAnalyzer(lang="en")
emotion_analyzer = EmotionAnalyzer(lang="en")

data = [['Hello world'], ['I am the best'], ['Nice jacket!']]
df2 = pd.DataFrame(data, columns = ['Tweet'])

# print dataframe.
df2["sentiment"] = df2.apply(lambda row : analyzer.predict(row["Tweet"]), axis = 1)

The output for the code below:

Tweet                       sentiment
---------------------|  --------------------
  Hello world        |  SentimentOutput(output=POS, probas={POS: 0.999, NEG: 0.001,NEU: 0.000})       |
  I am the best      |  SentimentOutput(output=POS, probas={POS: 0.999, NEG: 0.001,NEU: 0.000})
  Nice jacket!       | SentimentOutput(output=POS, probas={POS: 0.999, NEG: 0.001,NEU: 0.000})

I would like to split the sentiment column and have something like this:

Tweet                    sentiment    prob_Pos    Prob_Neg   Prob_Neu
---------------------|---------------|----------|------------------------------
  Hello world        |   POS         |  0.99    | 0.001    | 0.000
  I am the best      |   POS         |  0.99    | 0.001    | 0.000
  Nice jacket!       |   POS         |  0.99    | 0.001    | 0.000
pandas dataframe
1 Answers

The results must be converted into a pd.Series then join back to the DataFrame. This is easiest to do with a function as the results cannot be easily unpacked:

analyzer = SentimentAnalyzer(lang="en")


def process(row):
    res = analyzer.predict(row["Tweet"])
    return pd.Series({'sentiment': res.output, **res.probas})


df2 = df2.join(df2.apply(process, axis=1))

df2:

           Tweet sentiment       NEG       NEU       POS
0    Hello world       NEU  0.000446  0.548691  0.450863
1  I am the best       POS  0.000660  0.001529  0.997811
2   Nice jacket!       POS  0.000224  0.051520  0.948256

This can also be done in a way that the analyzer can be passed as a parameter:

def process_with(predictor):
    def process_(row):
        res = predictor.predict(row["Tweet"])
        return pd.Series({'sentiment': res.output, **res.probas})

    return process_


analyzer = SentimentAnalyzer(lang="en")
df2 = df2.join(df2.apply(process_with(analyzer), axis=1))
user_154978880
Related