Issue extracting "mentions" from Twitter data using regex

user_13337635 0
Viewed 147

I am trying to extract the mentions in a Tweet from Twitter i.e. @Google or @Apple.

This is my code so far to extract the mentions from a column and then create another column with the mentions.

df_bdtu['mentions'] = df_bdtu['tweet_text'].str.findall('(?:^|\s)[＠ @]{1}([^\s#<>[\]|{}]+)')

It works mostly but I am facing some issues with some edge cases for example take this Tweet:

Check out @Dreams_n_Songs and give them a follow! I can't recommend their hoodies enough!Shop now  …

The mentions stored in the mentions column below which is incorrect as it includes the emoji for some reason.

['Dreams_n_Songs', '…']

Another issue is when there is a . before the mentions such as this example:

.@ChelseaFC, @FCBayern, @VfL_Wolfsburg and more are among the latest names to be confirmed at -…

The mentions that are produced do not include the first mention.

[FCBayern,, VfL_Wolfsburg]

How would I fix the regex for this?

pandas regex
1 Answers

You can use

[＠@]([^][\s#<>|{}]+)

See the regex demo. So, remove (?:\s|^) that requires either start of string or a whitespace at the start of a match, and you need to remove a literal space from the [＠ @] character class.

In Pandas code, you can use it like this:

df_bdtu['mentions'] = df_bdtu['tweet_text'].str.findall(r'[＠@]([^][\s#<>|{}]+)')

Mind the r'...' raw string literal notation.

user_38329700
Related