How to detect multivariate outliers within large dataset?

We do have a special type of distance formula that we use to find multivariate outliers. It is called Mahalanobis Distance.

The MD is a metric that establishes the separation between a distribution D and a data point x by generalizing the z-score, the MD indicates how far x is from the D mean in terms of standard deviations.

You can use the below function to find out outliers. It returns the index of outliers.

from scipy.stats import chi2
import scipy as sp
import numpy as np
def mahalanobis_method(df):
    #M-Distance
    x_minus_mean = df - np.mean(df)
    cov = np.cov(df.values.T)                           #Covariance
    inv_covmat = sp.linalg.inv(cov)                     #Inverse covariance
    left_term = np.dot(x_minus_mean, inv_covmat) 
    mahal = np.dot(left_term, x_minus_mean.T)
    md = np.sqrt(mahal.diagonal())
    
    #Flag as outliers
    outliers = []
    #Cut-off point
    C = np.sqrt(chi2.ppf((1-0.001), df=df.shape[1]))    #degrees of freedom = number of variables
    for i, v in enumerate(md):
        if v > C:
            outliers.append(i)
        else:
            continue
    return outliers, md

If you want to study more about Mahalanobis Distance and its formula you can read this blog.

So, how to understand the above formula? Let’s take the (x – m)^T . C^(-1) term. (x – m) is essentially the distance of the vector from the mean. We then divide this by the covariance matrix (or multiply by the inverse of the covariance matrix). If you think about it, this is essentially a multivariate equivalent of the regular standardization (z = (x – mu)/sigma).