python3 counter that also averages index location in a sliding window

user_11984931 0
Viewed 46

I have a list and a string:

anchors=['a','b']
mystring = 'aerdsbewsabsdsasdbs'

I would like to iterate through each letter in mystring, and if I "hit" an anchor, keep track of the letters that occur 2 letters before "prior" and 2 letters after "post".

So my desired output would be a list of lists, in the form: 

[[prior letters, anchor, post letters],
prior letters, anchor, post letters]]

for each anchor. This can easily be achieved using a Counter() on the letters inside mystring, but I would like to sort the resulting output by the average index position.
So for each prior letter, I want to count how many times it occurs within 2 letters of each anchor, and sort by the average of the index position from the anchor. In my example for anchor 'a', this would look like: 

prior letters = {s:(2,-1), w:(1,-2), d:(1,-2)}
anchor = 'a'
post letters = {e:(1,1), r:(1,2), b:(1,1), s:(2,1.5), d:(1,2)}

where each key value pair for prior and post letters is in the form: 

letter:(count,avg_index)

letter = the letter inside the sliding window of 2
count = how many times the letter occurs in the sliding window for each anchor
avg_index = the average index position of the letter in relation to each anchor, for example (-1.5) would mean the letter occurs at positions -1 and -2, average of -1.5.
I'm struggling to complete the code, is there a better data structure I can use to help?

counter data-structures python-3.x python
2 Answers

not a full answer, but the comment would be too narrow. I've tested this code:

import pandas as pd

anchors=['a','b']
mystring = 'aerdsbewsabsdsasdbs'

df = pd.DataFrame(columns=['letter', 'match_anchor', '2_letters_before', '2_letters_afer'])
for letter in mystring:
    df = df.append( pd.DataFrame([[letter, letter in anchors]], columns=['letter', 'match_anchor']) )

df['2_letters_before'] = df['letter'].shift(2)
df['2_letters_afer'] = df['letter'].shift(-2)
df = df[df['match_anchor'] == True]
df = df.reset_index(drop=True)
print(df)

And the output is:

  letter match_anchor 2_letters_before 2_letters_afer
0      a         True              NaN              r
1      b         True                d              w
2      a         True                w              s
3      b         True                s              d
4      a         True                d              d
5      b         True                s            NaN

The point is I'm not understanding your data. With the current output, I guess I'm close to what you expect. However, could you explain more the expected output you want ? For example, what are the figures (2,-1) mean for the letter 's'/anchor 'a' ?

prior letters = {s:(2,-1), w:(1,-2), d:(1,-2)}
anchor = 'a'
post letters = {e:(1,1), r:(1,2), b:(1,1), s:(2,1.5), d:(1,2)}

EDIT: Well, not a very pythonic code, but I've managed to do it with pandas. First step: I'm constructing a DataFrame with the letters matching with anchors + letters within the window. Then, I construct a list of all the letters to look to. I do loop over theses letter to count them + compute the index. The ouput is not a dict, but I've only printed the values. It can easyly edited to match your needs.

import pandas as pd
import numpy as np

anchors=['a','b']
mystring = 'aerdsbewsabsdsasdbs'

# Building the DataFrame for calculations:
df = pd.DataFrame(columns=['letter', 'match_anchor'])
for letter in mystring:
    df = df.append( pd.DataFrame([[letter, letter in anchors]], columns=['letter', 'match_anchor']) )

df['2_let_bef'] = df['letter'].shift(2)
df['1_let_bef'] = df['letter'].shift(1)
df['1_let_aft'] = df['letter'].shift(-1)
df['2_let_aft'] = df['letter'].shift(-2)
df = df[df['match_anchor'] == True]
print(df)

# Getting the list of letters to look to:
let = pd.concat([df['2_let_bef'],
                 df['1_let_bef'],
                 df['2_let_aft'],
                 df['1_let_aft'],], ignore_index=True)
let = let.dropna().unique().tolist()
print('list of letters to look to:', let, '\n')

# looping through
for letter in anchors:
    print('\nAnchor=', letter)
    pf_anchor = df[df['letter'] == letter]
    # checking 'before'
    for l in let:
        count = len(pf_anchor[pf_anchor['2_let_bef'] == l]) + \
                len(pf_anchor[pf_anchor['1_let_bef'] == l])
        index_avg = -2*len(pf_anchor[pf_anchor['2_let_bef'] == l]) - \
                    len(pf_anchor[pf_anchor['1_let_bef'] == l])
        if count > 0:
            index_avg = index_avg/count
            print('(before)', l, ":", (count,index_avg))

    for l in let:
    # checking 'after'
        count = len(pf_anchor[pf_anchor['2_let_aft'] == l]) + \
                len(pf_anchor[pf_anchor['1_let_aft'] == l])
        index_avg = 2*len(pf_anchor[pf_anchor['2_let_aft'] == l]) + \
                    len(pf_anchor[pf_anchor['1_let_aft'] == l])
        if count > 0:
            index_avg = index_avg/count
            print('(after)', l, ":", (count,index_avg))

output:

Anchor= a
(before) d : (1, -2.0)
(before) w : (1, -2.0)
(before) s : (2, -1.0)
(after) d : (1, 2.0)
(after) s : (2, 1.5)
(after) r : (1, 2.0)
(after) e : (1, 1.0)
(after) b : (1, 1.0)

Anchor= b
(before) d : (2, -1.5)
(before) s : (3, -1.6666666666666667)
(before) a : (1, -1.0)
(after) d : (1, 2.0)
(after) w : (1, 2.0)
(after) s : (2, 1.0)
(after) e : (1, 1.0)
user_134813770

I was able to write it without pandas/numpy for a more pythonic answer, but accepted Oliviers as he got to it first and it works. 

from collections import Counter, defaultdict
from statistics import mean
mytext = 'sldksfjksljdkljasflkjsfdjkldfslkjsfdsaljkdfsljkbfdsjlkdfs'
anchors = ['k','s']
out = []
letters = []
amount=0
avgindex=0
for anchor in anchors:
    anchordict = {anchor:(amount,avgindex)}
    letterindex = defaultdict(list)
    for i,letter in enumerate(mytext):
        if anchor==letter:
            z=-2
            for letter in mytext[i-2:i]:
                letterindex[letter].append(z)
                z=z+1
            anchordict[anchor] = (anchordict[anchor][0]+1,anchordict[anchor][1]+1)
    out.append(anchordict)
    for k,v in letterindex.items():
        letterindex[k] = mean(v)
    for k,v in letterindex.items():
        letters.append([anchor,k,v])
letters.sort(key=lambda tup: tup[2])
print(out)
print(letters)
user_119849310
Related