Counter() and plot the most common word in a text

Question

I have written a function that outputs and plots the most common words found in the text. Please see the code below and the output.

tf = Counter()
for i  in list(tweet['text']):
    temp=XXX 
for tag, count in tf.most_common(20):
        print("{}: {}".format(tag, count))   
        
y = [count for tag, count in tf.most_common(20)]
x = range(1, len(y)+1)

plt.bar(x, y)
plt.title("Term frequencies used inTwitter Data")
plt.ylabel("Frequency")
plt.savefig('us-iran-term-distn.png')

the Output is the most frequent words with a plot below:

blacklivesmatter: 127336
blm: 58619
black: 25973
people: 17960
.
.
lives: 11684
police: 10762
matter: 9902
white: 9766
georgefloyd: 9023
protest: 8734

How can i add in the x-axis the most frequent words please?

Thanks very much

user_12046409 · Answer

You can directly use the list of tags for the x-values. Matplotlib will display these texts as the ticklabels of the axis. Optionally, use can use plt.yscale('log') to better distinguish the lower values.

The code below first generates a random list of words, following a zipf distribution.

from collections import Counter
import numpy as np
from matplotlib import pyplot as plt

words = ['Red', 'Orange', 'Yellow', 'Green', 'Blue', 'Purple', 'Brown', 'Magenta', 'Tan', 'Cyan', 'Olive', 'Maroon', 'Navy', 'Aquamarine', 'Turquoise', 'Silver', 'Lime', 'Teal', 'Indigo', 'Violet', 'Pink', 'Black', 'White', 'Gray']
indices = np.random.zipf(1.6, size=100000).astype(np.int) % len(words)
tweets = np.array(words)[indices]

tf = Counter(tweets)

y = [count for tag, count in tf.most_common(20)]
x = [tag for tag, count in tf.most_common(20)]

plt.bar(x, y, color='crimson')
plt.title("Term frequencies in Twitter Data")
plt.ylabel("Frequency (log scale)")
plt.yscale('log') # optionally set a log scale for the y-axis
plt.xticks(rotation=90)
for i, (tag, count) in enumerate(tf.most_common(20)):
    plt.text(i, count, f' {count} ', rotation=90,
             ha='center', va='top' if i < 10 else 'bottom', color='white' if i < 10 else 'black')
plt.xlim(-0.6, len(x)-0.4) # optionally set tighter x lims
plt.tight_layout() # change the whitespace such that all labels fit nicely
plt.show()

Counter() and plot the most common word in a text

1 Answers