Matrix from bigram?

user_13915556 0
Viewed 213

I have a list of chords

['F', 'Em7', 'A7', 'Dm', 'Dm7', 'A#', 'C7', 'C', 'G7', 'A7sus4', 'Gm6', 'Fsus4', etc]

and I want to turn it into a transition matrix. What I have so far:

import pandas as pd
from nltk.util import ngrams

bigram = pd.Series(ngrams(data.split(),2)) ### creating the bigrams of ordered pairs like:
### 0      (F, Em7) 
### 1     (Em7, A7) 
### 2      (A7, Dm)

probabilities = bigram.value_counts(normalize=True) ### getting probability of each ordered pair
letters = pd.Series(data.split()).unique() ### getting each chord

I want the matrix to be like:

          F        Em7      A7          ....
 F     p(FF)    p(FEm7)    p(FA7)       ....
 Em7   p(Em7F)  p(Em7Em7)  p(Em7A7)     ....
 A7    p(A7F)    p(A7Em7)  p(A7A7)      ....
  .
  .
  .

where p(FF) is the probability of (F, F) which is in my probabilities list. How can I do this? Thank you!!

pandas probability nltk python
1 Answers

Create a new pd.Series having the MultiLevel index created using ordered pairs of bigrams from prob list, then use Series.unstack with fill_value=0 to reshape the series into a pivoted datafarme, optionally round the values.:

prob = bigram.value_counts(normalize=True)

mat = (
    pd.Series(
        prob, index=[prob.index.str[0], prob.index.str[1]])
    .unstack(fill_value=0).round(3)
)

Result:

print(mat)

           A#     A7  A7sus4      C     C7     Dm    Dm7    Em7  Fsus4     G7    Gm6
A#      0.000  0.000   0.000  0.000  0.091  0.000  0.000  0.000  0.000  0.000  0.000
A7      0.000  0.000   0.000  0.000  0.000  0.091  0.000  0.000  0.000  0.000  0.000
A7sus4  0.000  0.000   0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.091
C       0.000  0.000   0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.091  0.000
C7      0.000  0.000   0.000  0.091  0.000  0.000  0.000  0.000  0.000  0.000  0.000
Dm      0.000  0.000   0.000  0.000  0.000  0.000  0.091  0.000  0.000  0.000  0.000
Dm7     0.091  0.000   0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000
Em7     0.000  0.091   0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000
F       0.000  0.000   0.000  0.000  0.000  0.000  0.000  0.091  0.000  0.000  0.000
G7      0.000  0.000   0.091  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000
Gm6     0.000  0.000   0.000  0.000  0.000  0.000  0.000  0.000  0.091  0.000  0.000
user_128331660
Related