How to forward fill a column subset with method chaining?

user_10680954 0
Viewed 125

I have a dataframe that contains information about days. Each day has two phases.

df = (pd.DataFrame(dict(
    day=[0, 1, 2],
    phase1=[1, 0, 2],
    phase2=[5, 0, 7],
))

The goal is to forward fill days on which both phases are 0. The expected output looks like this

out_wanted = pd.DataFrame(dict(
    day=[0, 1, 2],
    phase1=[1, 1, 2],
    phase2=[5, 5, 7],
))

I am searching for a solution to this problem that works with method chaining. The syntax should somewhat look like this:

out_wanted = (
    df
    .methodA(...)
    .methodB(...)
    .forward_fill_magic()
    .methodC(...)
    .methodD(...)
)

How can I do that?

Update:

Let's imagine df has two columns named day and x that can contain zeros and I want to avoid replacing any values in them with nan. How would I do that?

df = pd.DataFrame(dict(
    day=[0, 1, 2],
    phase1=[1, 0, 2],
    phase2=[5, 0, 7],
    x=[1, 2, 3]
))

The current proposed solution gives me an ValueError

df = (
    df.set_index(["day", "x"])
    .mask(df.eq(0))
    .ffill(downcast= 'infer')
)

-> ValueError: cannot join with no overlapping index names

pandas method-chaining python-3.x python
2 Answers

Here is a solution without setting (and later resetting) an index and with an explicit selection of columns to be filled. It also fills only if both columns are equal to 0 (which I think was part of the requirements).

def forward_fillmagic(df, colnames):
    all_zero = df[colnames].eq(0).all(axis=1)
    return df.assign(**{
        colname: df[colname].mask(all_zero).ffill(downcast="infer")
        for colname in colnames
    })

df.pipe(forward_fillmagic, ["phase1", "phase2"])
user_14883830

TRY with set_index /mask /ffill :

df = df.set_index(["day", "x"]).mask(lambda d: d.eq(0)).ffill().reset_index()

ALTERNATIVE_1 :

df = (
    df.set_index(["day", "x"])
    .replace(0,np.nan)
    .ffill(downcast= 'infer')
)

ALTERNATIVE_2 :

df[df.filter(like = 'phase').columns] = df.filter(like= 'phase').mask(df.eq(0)).ffill(downcast='infer')

OUTPUT:

     phase1  phase2
day                
0         1       5
1         1       5
2         2       7
user_154380330
Related