How can I get unique words for each speaker and the counts?

ADITYA YADAV 0
Viewed 48
WILL: I’ve never seen wildlings do a thing like this. I’ve never seen a thing like this, not ever in my life.

WAYMAR ROYCE: How close did you get?

WILL: Close as any man would.

GARED: We should head back to the wall.

ROYCE: Do the dead frighten you?

GARED: Our orders were to track the wildlings. We tracked them. They won’t trouble us no more.

I want to get unique words spoken by each speaker and the counts

with open ("conv.txt", "r") as hfile:
    sp = hfile.read()
    print (sp)

dictionary = {}
for x, line in enumerate(sp):
    line_list = sp.split(":")
    dictionary[line_list[0]]=line_list[1]

I tried this this code but not working.

The output I want:

unique words spoken by each speaker WILL,GARED,ROYCE

python
2 Answers

I would use collections.Counter and a single loop here:

from collections import Counter

c = Counter()

with open ("conv.txt", "r") as hfile:
    for line in hfile:                         # for each line
        speaker, *text = line.split(":", 1)    # split as speaker and text
        if text:                               # if there is text (i.e. not a blank line)
            c[speaker] += len(text[0].split()) # add number of words to counter

print(dict(c))

NB. counting "words" as any sequence of non space characters here. This might not be exactly what you want. In such case better use a dedicated library (e.g. nltk) to identify the words.

output:

{'WILL': 26, 'GARED': 23, 'WAYMAR ROYCE': 5, 'ROYCE': 5}
mozway0

As @mozway points out, we can use a Counter for this, but we'll use a Counter for each line.

from collections import defaultdict, Counter

d = defaultdict(Counter)
with open("conv.txt") as hfile:
    for line in hfile:
        if not line:
            continue # if the line is blank move to the next one
        speaker, text = line.split(":", 1)
        # this is where the core logic is. we update the speaker
        # with the counts of each line. `text.split()` splits the lines
        # on spaces. we could also normalise a bit here by making `text`
        # lowercase and removing punctuation.
        d[speaker] += Counter(text.split())

import pprint
pprint.pprint(dict(d))  # this just gives better formatting and makes it into a normal dict

Which gets this:

{'GARED': Counter({'We': 2,
                   'to': 2,
                   'the': 2,
                   'should': 1,
                   'head': 1,
                   'back': 1,
                   'wall.': 1,
                   'Our': 1,
                   'orders': 1,
                   'were': 1,
                   'track': 1,
                   'wildlings.': 1,
                   'tracked': 1,
                   'them.': 1,
                   'They': 1,
                   'won’t': 1,
                   'trouble': 1,
                   'us': 1,
                   'no': 1,
                   'more.': 1}),
 'ROYCE': Counter({'Do': 1, 'the': 1, 'dead': 1, 'frighten': 1, 'you?': 1}),
 'WAYMAR ROYCE': Counter({'How': 1, 'close': 1, 'did': 1, 'you': 1, 'get?': 1}),
 'WILL': Counter({'I’ve': 2,
                  'never': 2,
                  'seen': 2,
                  'a': 2,
                  'thing': 2,
                  'like': 2,
                  'wildlings': 1,
                  'do': 1,
                  'this.': 1,
                  'this,': 1,
                  'not': 1,
                  'ever': 1,
                  'in': 1,
                  'my': 1,
                  'life.': 1,
                  'Close': 1,
                  'as': 1,
                  'any': 1,
                  'man': 1,
                  'would.': 1})}
blueteeth0
Related