Have number of words per sentence, how to get mean?

user_11330134 0
Viewed 132

I have a function and want to return (a) the number of words per sentence and (b) mean length of words per sentence in a list of tuples. I can get (a). For (b) I can get the total number of characters per sentence but not mean.

I've reviewed a few posts (this, that and another) but can't wrap my head around this last piece.

I've included a couple failed attempts commented out.

import statistics

def sentence_num_and_mean(text):
    """ Output list of, per sentence, number of words and mean length of words """
    # Replace ! and ? with .
    for ch in ['!', '?']:
        if ch in text:
            text = text.replace(ch, '.')

    # Number of words per sentence
    num_words_per_sent =  [len(element) for element in (element.split() for element in text.split("."))]

    # Mean length of words per sentence

    # This gets sum of characters per sentence, so on the right track
    mean_len_words_per_sent = [len(w) for w in text.split('.')]

    # This gives me "TypeError: unsupported operand type(s) for /: 'int' and 'list'" error
    # when trying to get the denominator for the mean
    # A couple efforts
    #mean_len_words_per_sent = sum(num_words_per_sent) / [len(w) for w in text.split('.')]
    #mean_len_words_per_sent = [(num_words_per_sent)/statistics.mean([len(w) for w in text.split()])]

    # Return list zipped together
    return list(zip(num_words_per_sent, mean_len_words_per_sent))

Driver program:

split_test = "First sentence ends with a period. Next one ends with a question mark? Another period. Then exclamation! Blah blah blah"
func_test = sentence_num_and_mean(split_test)
print(split_test)
print(func_test)

which prints

First sentence ends with a period. Next one ends with a question mark? Another period. Then exclamation! Blah blah blah
[(6, 33), (7, 35), (2, 15), (2, 17), (3, 15)]

For one, I need to strip out spaces and periods, but ignoring that for now, if I did the simple math right it should be:

[(6, 5.5), (7, 5), (2, 7.5), (2, 8.5), (3, 5)]
python
5 Answers

Better variable names may help you clarify how to express the ideas. text.split('.') gives you what? A list of sentences (str). If you have a single sentence in a variable called sentence then sentence.split() gives you a list of words (str). With those in mind, this is pretty easy to write.

mean_len_words_per_sent = [statistics.mean(len(word) for word in sentence.split()) for sentence in text.split('.')]

user_21422480

It's pretty clear that what you're ending up with is the correct number of words per sentence and the expected number of characters per word (before removing whitespace and punctuation). So all you need is the former divided by the latter.

num_words_per_sent =  [len(element) for element in (element.split() for element in text.split("."))]

len_words_per_sent = [len(w) for w in text.split('.')]

return [(num,len_words/num) for num,len_words in zip(num_words_per_sent,len_words_per_sent)]
user_130321290

The list mean_len_words_per_sent should probably be num_characters_per_sent as it is currently used.

You can then iterate through the two lists you created and divide the character per sentence through the number of words in the sentence.

mean_len_words_per_sent = [num_chars / num_word for num_chars, num_word in zip(num_characters_per_sent, num_words_per_sent)]
user_41137210

If you only want letters, this should work:

def sentence_num_and_mean(text):
    # Replace ! and ? with .
    for ch in ['!', '?']:
        if ch in text:
            text = text.replace(ch, '.')

    output = []
    sentences = text.split(".")
    for sentence in sentences:
        words = [x for x in sentence.split(" ") if x]
        word_count = len(words)
        word_length = sum(map(len, words))
        word_mean = word_length / word_count
        output.append((word_count, word_mean))

    return output


split_test = "First sentence ends with a period. Next one ends with a question mark? Another period. Then exclamation! Blah blah blah"
func_test = sentence_num_and_mean(split_test)
print(split_test)
print(func_test)

Output:

First sentence ends with a period. Next one ends with a question mark? Another period. Then exclamation! Blah blah blah
[(6, 4.666666666666667), (7, 4.0), (2, 6.5), (2, 7.5), (3, 4.0)]
user_85209830

You can use statistics.mean for computing the average word length. Here you can use map(len, sentence.split()) in order to compute the length of each word.

import statistics

def sentence_num_and_mean(text):
    punctuation = '?!'
    text = text.translate(str.maketrans(dict.fromkeys(punctuation, '.')))
    sentences = text.split('.')
    num_words_per_sent = [len(s.strip().split()) for s in sentences]
    mean_len_words_per_sent = [statistics.mean(map(len, s.strip().split())) for s in sentences]
    return list(zip(num_words_per_sent, mean_len_words_per_sent))
user_37672390
Related