Is BigramCollocationFinder with BigramAssocMeasures.likelihood_ratio non-deterministic?

Viewed 16

I am attempting to use BigramCollocationFinder for a set of sentences (500 sentences).

Here is my python code logic:

words = []
#sentences = ["my name is...","i live in..."]
for sentence in sentences:
    words.extend(sentence.strip().split(' '))
bigrams = nltk.collocations.BigramAssocMeasures()
finder = BigramCollocationFinder.from_words(words)
finder.apply_freq_filter(3)
outputs = finder.nbest(bigrams.likelihood_ratio, 100)

Now, I see 2 sorts of behaviour:

  1. If I keep the order of sentences in the 'sentences' array fixed and run the code N times, I get different number of bigrams every time
  2. If I change the order of sentences in the 'sentences' array and run the code N times, then also I get different number of bigrams every time

My questions:

  1. Why is bigram finder behaving non-deterministically? Is it due the usage of likelihood_ratio?

  2. How can it be made deterministic for a fixed input array of sentences?

0 Answers
Related