efficient way to convert pandas series of strings to numpy frequency matrix

user_7972317 0
Viewed 64

I have:

  1. a list of english chars, a to z: char_list
  2. series of strings

I want to create a numpy matrix where each row correspond to the string at the same row in the series, and each column to the character at the same index in the list.

example:

series: [[ab],[ac],[aa]]
chars = [a,b,c]
result = [[110],[101],[200]]

here is how I do it now:

def create_char_matrix(strings, symbol_list):
    
    mat = np.zeros((strings.shape[0],len(symbol_list)))
    
    for i, line in enumerate(strings):
        for c in line:
            mat[i,symbol_list.index(c)] += 1 
 
    
    return mat

this is not very fast, considering there is often a better solution than nested for loops.

any ideas about how to accelerate this process?

pandas numpy python
1 Answers

You can split the string to the characters, then crosstab:

s = pd.Series(['ab','ac','aa'])
chars=['a','b','c']
    
a = s.str.split('').str[1:-1].explode()
pd.crosstab(a.index, a).reindex(chars, axis=1, fill_value=0).values

Output:

array([[1, 1, 0],
       [1, 0, 1],
       [2, 0, 0]])
user_42384080
Related