Density plot for many samples showing overall trend - how?

Question

I'd like to show a density plot for many samples. Each sample belongs to a particular grouping variable. I can plot each individual density plot like so:

import seaborn as sns
fmri = sns.load_dataset("fmri")[['subject','timepoint','region','signal']].drop_duplicates(['subject','timepoint','region'])

region2col={'parietal':'red', 'frontal':'blue'}
fig, ax= plt.subplots(figsize=(22,10))
for subject in fmri.subject.unique():
  temp=fmri.loc[fmri.subject==subject,]
  for region in temp['region'].unique():
    temp2=temp.loc[temp.region==region,]
    
    sns.distplot(
      temp2['signal'],
      label = region,
      color=region2col[region],
      kde=True, hist=False,
      ax=ax
      )

However, I'd like to draw instead an overall density of the distribution of each region (same axes as above, signal and density) but with a shaded area for extremes (maximum and minimum at each signal point) and an overall fitting line describing the general trend. Similar to this:

#example only to show formatting wanted.
# XX axis should show "signal"
# YY axis should show density
g = sns.relplot(x="timepoint", y="signal",
                hue="region",
                kind="line", data=fmri)
plt.show()

Is this possible?

user_12046409 · Answer

Scipy's gaussian_kde can be used to calculate all the individual kdes. sns.lineplot (the function that is called by sns.relplot to draw line plots) creates confidence intervals when the same x-values correspond to multiple y-values. So repeating an array of x values, once for every array of corresponding kde values, enable the desired plot.

import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
from scipy.stats import gaussian_kde

fmri = sns.load_dataset("fmri")[['subject', 'timepoint', 'region', 'signal']].drop_duplicates(
    ['subject', 'timepoint', 'region'])
region2col = {'parietal': 'red', 'frontal': 'blue'}
x_min = fmri['signal'].min()
x_max = fmri['signal'].max()
dx = 0.1 * (x_max - x_min)
x_min -= dx
x_max += dx
xs = np.linspace(x_min, x_max, 100)
fig, ax = plt.subplots(figsize=(12, 5))
for region in fmri['region'].unique():
    df_region = fmri.loc[fmri.region == region,]
    kdes = []
    for subject in fmri.subject.unique():
        signal = df_region.loc[(df_region.subject == subject),]['signal']
        kde = gaussian_kde(signal)
        kdes.append(kde(xs))
    num_subjects = len(kdes)
    kdes = np.concatenate(kdes)
    sns.lineplot(x=np.tile(xs, num_subjects), y=kdes, label=region, color=region2col[region], ax=ax)
    ax.set_xlabel('signal')
    ax.set_ylabel('density')
plt.legend(title='Region')
plt.show()

user_1356000 · Answer

This is probably not the fastest method, but you could calculate the kde for each subject/region over a certain range, and then let lineplot do the rest

from scipy.stats import gaussian_kde
x = np.linspace(fmri['signal'].min(),fmri['signal'].max(),100)
temp = fmri.groupby(['subject','region'])['signal'].apply(lambda temp: pd.Series(gaussian_kde(temp).evaluate(x), index=pd.Index(x, name='x')))
temp = temp.reset_index(name='kde')

plt.figure()
sns.lineplot(data=temp, x='x', y='kde', hue='region')

Density plot for many samples showing overall trend - how?

2 Answers