What is the write way to assign new values to a slice of dataset and return back the whole dataset

user_8176763 0
Viewed 47

I have a pandas dataframe that I want to transform some columns and assign back the transformation into the existing dtaframe:

df.loc[:,df.columns != 'target'].pct_change(-1)

If I do:

df.loc[:,df.columns != 'target'] = df.loc[:,df.columns != 'target'].pct_change(-1)

I get a warning:

A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

What is the right to way to get back the transformed df ? thanks

pandas dataframe python
3 Answers

You can do update

df.update(df.loc[:,df.columns != 'target'].pct_change(-1))
user_79645270

Filter out columns that have word target in them using df.filter

df[df.filter(regex='[^target]').columns]=df.loc[:,df.filter(regex='[^target]').columns].pct_change(-1)
user_89869750

After some research I think I found the best way to do this:

df.loc[:,df.columns != 'target'] = df[df.columns[:-1]].pct_change(-1).to_numpy()

According to pandas https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#basics the best way to apply a transform in place to a subset of columns is by using raw values, since my column was the last one I decided to not use .loc on the right side of the = sign but rather index out the last element of the list, that way I get no warnings.

user_81767630
Related