Python. Get uninterrupted length of segments along rows for each column

user_5310300 0
Viewed 59

I have a file like this one, with segments of 0s, 1s and 2s:

0000000001111100110002220000000011111111
0011100000111000000220111110001111100000
1110011111111111000001111000002222111000
0011110001110000000220220000001111100000

For example, row 1 has a segment of 9 0s, followed by a segment of 5 1s, followed by a segment of 2 0s, and so on. In this toy example, each row goes from position 0 to 39 (Python), but in reality it goes from 0 to >2000000 and there are >10000 rows. For each column, I would like to get the length of uninterrupted segments of 0s, 1s and 2s. For example, for: 

column 0, level 0, I have: 9, 2, 0 and 2. 
column 0, level 1, I have: 0, 0, 3 and 0. 
column 0, level 2, I have: 0, 0, 0 and 0. 

For column 10, level 0: 0,5,0 and 0
For column 10, level 1: 5,0,11 and 3
For column 10, level 2: 0,0,0 and 0

For column 23, level 0: 0,0,0 and 0
For column 23, level 1: 0,5,4 and 0
For column 23, level 2: 3,0,0 and 2

At the end, I will like to have 3 different matrices (for level 0,1 and 2) with the length of the segments for each column.

For 0, it will look something like this:

9.........0............0.................
2.........0............0.................
0.........0............0.................
2.........0............0.................

For 1:

0.........5............0.................
0.........3............0.................
3.........11............0.................
0.........3............0.................

For 2:

0.........0............3.................
0.........0............0.................
2.........0............0.................
0.........0............2.................

How to do this with Python numpy or pandas?

matrix python
1 Answers

Assuming you could read the file into a list of strings (using readlines for instance), you could create a dataframe of tuples of (level, count) using itertools.groupby and then iterate over those to get the desired result, here goes: 

import itertools
import pandas as pd
import numpy as np

l = \
["0000000001111100110002220000000011111111",
"0011100000111000000220111110001111100000",
"1110011111111111000001111000002222111000",
"0011110001110000000220220000001111100000"]

def level_and_count(row):
    g1 = [(x[0], len(list(x[1]))) for x in itertools.groupby(row)]
    return sum([[x]*x[1] for x in g1], [])

df = pd.DataFrame([level_and_count(row) for row in l])

def mat_by_level(df, level):
    aa = df.apply(lambda c: [x[1] if x[0]==level else 0 for x in c],   axis=1)
    return pd.DataFrame(np.vstack(aa))

and now, when we have the functions, we can run them on each level and save the result to a file (comma separated in this example):

for level in ['0', '1', '2']:
    mat_by_level(df, level).to_csv(f'level_{level}.csv', index=False, header=False)

let me know if this is what you meant.

user_49089000
Related