populate dataframe with loop

user_16671648 0
Viewed 136

I'm new into python but and already spent a lot of time without any visible progress on this issue. hope someone could direct me on my question! i have some track data with samples collected.

track direction samples
1 forward 11
1 backward 10
2 forward 40
2 backward 30

and i need to populate dataframe with multiplied number of rows for each direction for each track by number in the column "samples"

the output should be something like that:

track direction sample
1 forward 1
1 forward 2
1 forward 3
1 forward ...
1 forward 11
1 backward 1
1 backward ...
1 backward 10
2 forward 1
2 forward ...
2 forward 40
2 backward 1
2 backward ...
2 backward 30

thank you in advance!

pandas dataframe python
2 Answers

Apply range over your sample and explode the list and join to other columns.

df = df[['track', 'direction']].join(
        df['samples'].apply(range)
                     .explode()
                     .add(1)
                     .rename('sample')
     )

>>> df
    track direction sample
0       1   forward      1
0       1   forward      2
0       1   forward      3
0       1   forward      4
0       1   forward      5
..    ...       ...    ...
3       2  backward     26
3       2  backward     27
3       2  backward     28
3       2  backward     29
3       2  backward     30

[91 rows x 3 columns]

The solution of @ThePyGuy is valid too:

df.assign(sample=df['samples'].add(1)
                              .apply(lambda x: range(1,x))) \
  .explode('sample') \
  .drop(columns=['samples'])

Performance

%timeit df[['track', 'direction']].join(df['samples'].apply(range).explode().add(1).rename('sample'))
2.75 ms ± 356 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

%timeit df.assign(sample=df['samples'].add(1).apply(lambda x: range(1,x))).explode('sample').drop(columns=['samples'])
5.65 ms ± 308 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

It's not significant for this sample, however both operations are slow.

user_152399510

Here is another way:

(df.loc[df.index.repeat(df['samples'])]
 .assign(samples = lambda x: x.groupby(level=0).cumcount() + 1))
user_138021150
Related