Calculate standard deviation and mean of column for each row excluding the row for which mean is calculated

user_13300489 0
Viewed 40

I need to calculate the mean and standard deviation of a certain column in DataFrame so that the mean and standard deviation for each row is calculated excluding the value of the row for which it's calculated.

I know I can iterate each row by index, dropping each row by index in every iteration, and then calculate the mean. I wonder if there's a more efficient way of doing it.

df=pd.DataFrame({'marks':[10,20,30,10]})

I have one solution which works only when no element is repeating but in my data, we have 10 repeating two times so my approach doesn't work.

Approach:

data['marks'].apply(lambda x: data.loc[data.marks!= x, 'marks'].mean())

data['marks'].apply(lambda x: np.std(data.loc[data.marks!= x, 'marks']))

Can someone help me with this, please?

pandas python
1 Answers

The average is the sum of all values divided the amount of elements, therefore to exclude considering the own row's value we can try:

def mean_except_self(s):
    '''
    calculate mean of the series except the value of each row

    '''
    if len(s) <= 1: return np.nan
    return (s.sum() - s) / (len(s) - 1)

data['new_average'] = mean_except_self(data['marks'])
data['new_std'] = mean_except_self(data['marks']**2) - data['new_average']**2
user_118970070
Related