Changing values of one column based on the other three columns in pandas dataframe

Viewed 64

I have a following Pandas dataframe, where I want to change a value of 'fmc' column based on 'time', 'samples' and 'uid' columns.

Concept is as following:

For the same date, if df.samples == 'C' & df.uid == 'Plot1', then corresponding row value of fmc * 0.4

similarly for the same date, if df.samples == 'C' and df.uid == 'Plot2', then corresponding row value of fmc*0.8

For the same date, if df.samples == 'E' & df.uid == 'Plot1', then corresponding row value of fmc * 0.4

similarly for the same date, if df.samples == 'E' and df.uid == 'Plot2', then corresponding row value of fmc*0.15

For the same date, if df.samples == 'ns' & df.uid == 'Plot1', then corresponding row value of fmc * 0.2

similarly for the same date, if df.samples == 'ns' and df.uid == 'Plot2', then corresponding row value of fmc*0.05

I am new to python, so I apologize if I couldn't explain well and please let me know if you need more clarification.

       time        samples    uid                 fmc
0  2015-10-11        C       Plot1              98.226352
1  2015-10-11        C       Plot2             132.984817
2  2015-10-11        E       Plot1             114.147964
3  2015-10-11        E       Plot2             110.083699
4  2015-10-11        ns      Plot1             113.258977
5  2015-10-11        ns      Plot2             113.768023
6  2015-10-19        C       Plot1             118.503214
7  2015-10-19        E       Plot1             108.733209
8  2015-10-19        ns      Plot1              59.316977
9  2015-10-27        C       Plot1             104.977531
10 2015-10-27        C       Plot2             121.213887
11 2015-10-27        E       Plot1             129.575670
12 2015-10-27        E       Plot2             118.639048
13 2015-10-27        ns      Plot1             103.581065
14 2015-10-27        ns      Plot2             102.278469
15 2015-11-17        C       Plot1             103.820689
16 2015-11-17        C       Plot2             117.333382
17 2015-11-17        E       Plot1             143.418932
18 2015-11-17        E       Plot2             160.342155
19 2015-11-17        ns      Plot1              89.890484
4 Answers

This code:

import pandas as pd
data = [
    ['2015-10-11', 'C', 'Plot1',  98.226352 ],
    ['2015-10-11', 'C', 'Plot2', 132.984817 ],
    ['2015-10-11', 'E', 'Plot1', 114.147964 ],
    ['2015-10-11', 'E', 'Plot2', 110.083699 ],
    ['2015-10-11', 'ns', 'Plot1', 113.258977 ],
    ['2015-10-11', 'ns', 'Plot2', 113.768023 ],
    ['2015-10-19', 'C', 'Plot1', 118.503214 ],
    ['2015-10-19', 'E', 'Plot1', 108.733209 ],
    ['2015-10-19', 'ns', 'Plot1',  59.316977 ],
    ['2015-10-27', 'C', 'Plot1', 104.977531 ],
    ['2015-10-27', 'C', 'Plot2', 121.213887 ],
    ['2015-10-27', 'E', 'Plot1', 129.575670 ],
    ['2015-10-27', 'E', 'Plot2', 118.639048 ],
    ['2015-10-27', 'ns', 'Plot1', 103.581065 ],
    ['2015-10-27', 'ns', 'Plot2', 102.278469 ],
    ['2015-11-17', 'C', 'Plot1', 103.820689 ],
    ['2015-11-17', 'C', 'Plot2', 117.333382 ],
    ['2015-11-17', 'E', 'Plot1', 143.418932 ],
    ['2015-11-17', 'E', 'Plot2', 160.342155 ],
    ['2015-11-17', 'ns', 'Plot1',  89.890484]
]

df = pd.DataFrame(columns=['time', 'samples', 'uid', 'fmc'], data=data)

print (df.head(10))

df['result'] = df.apply(
                lambda item:
                   (item.fmc * 0.4) if item.samples == 'C' and item.uid == 'Plot1' else \
                   (item.fmc * 0.8) if item.samples == 'C' and item.uid == 'Plot2' else \
                   (item.fmc * 0.4) if item.samples == 'E' and item.uid == 'Plot1' else \
                   (item.fmc * 0.15)if item.samples == 'E' and item.uid == 'Plot2' else \
                   (item.fmc * 0.2) if item.samples == 'ns'and item.uid == 'Plot1' else \
                   (item.fmc * 0.05)if item.samples == 'ns'and item.uid == 'Plot2' else None,
                axis=1
            )

print(df.head(10))

Should produce this output:

         time samples    uid         fmc
0  2015-10-11       C  Plot1   98.226352
1  2015-10-11       C  Plot2  132.984817
2  2015-10-11       E  Plot1  114.147964
3  2015-10-11       E  Plot2  110.083699
4  2015-10-11      ns  Plot1  113.258977
5  2015-10-11      ns  Plot2  113.768023
6  2015-10-19       C  Plot1  118.503214
7  2015-10-19       E  Plot1  108.733209
8  2015-10-19      ns  Plot1   59.316977
9  2015-10-27       C  Plot1  104.977531
         time samples    uid         fmc      result
0  2015-10-11       C  Plot1   98.226352   39.290541
1  2015-10-11       C  Plot2  132.984817  106.387854
2  2015-10-11       E  Plot1  114.147964   45.659186
3  2015-10-11       E  Plot2  110.083699   16.512555
4  2015-10-11      ns  Plot1  113.258977   22.651795
5  2015-10-11      ns  Plot2  113.768023    5.688401
6  2015-10-19       C  Plot1  118.503214   47.401286
7  2015-10-19       E  Plot1  108.733209   43.493284
8  2015-10-19      ns  Plot1   59.316977   11.863395
9  2015-10-27       C  Plot1  104.977531   41.991012

Process finished with exit code 0

Inspired by df.apply, using axis=1, and passing a lambda function containing full set of the conditions, you will have the expected values in result column.

The apply function will pass the dataframe's columns (because axis=1), to the lambda function as item for each record in the series of values. The lambda function also, returns the corresponding result value for each given record/item in the series, so we don't need to worry about matching date/index values.

Reference for pandas.DataFrame.apply here.

Create queries

C1=(df.samples.eq('C')&df.uid.eq('Plot1'))|(df.samples.eq('E')&df.uid.eq('Plot1'))
C2=df.samples.eq('C')&df.uid.eq('Plot2')

C4=df.samples.eq('E')&df.uid.eq('Plot2')
C5=df.samples.eq('ns')&df.uid.eq('Plot1')
C6=C5=df.samples.eq('ns')&df.uid.eq('Plot2')

Put queries in a list

conditions=[C1,C2,C4,C5,C6]

Enlist outcome corresponding each query in list

MULT=[0.4,0.8,0.15,0.2,0.05]

Create temporary column and populate using np.select(condition, outcome)

df['fmc1']=np.select(conditions, MULT,df.fmc)

Multiply outcome with fmc and drop temporary column

df=df.assign(fmc=df['fmc']*df['fmc1']).drop('fmc1',1)

Make a function that takes features (columns) and returns result based on conditions.

def arrange_stuff(col2, col3, col4):
   if col2 == 'C' & col3 == 'Plot1'
      return col4*0.4
   elif ...
      return ...

Then make a new feature by applying the function as follows:

df['fmc_new'] = df(lambda x: arrange_stuff(x['samples'],x['uid'],x['fmc']), axis=1)

If you don't need the original fmc column, you can simply drop it and rename fmc_new, or maybe assign directly to it in the first place.

You should be able to use itertuples() to handle this one, which allow you to go through the rows of a data frame. I'm not sure what you mean by "for the same date", considering that your criteria for samples and uid cover all dates.

fmc_adjusted = []
for row in df.itertuples():
    if df.samples == 'C' and df.uid == 'Plot1':
        fmc_adjusted.append(row[4]*0.4)
    if df.samples == 'C' and df.uid == 'Plot2':
        fmc_adjusted.append(row[4]*0.15)

... and so on for your different criteria.

I like to keep my columns just in case I need to reference them later. If you want to create a new column:

df['fmc_adjusted'] = fmc_adjusted

If you want to replace your fmc column:

df['fmc'] = fmc_adjusted

There are probably quicker and neater ways of doing this, but I am not aware.

Related