Fill nan values after applying gaussian function

Question

I am applying from scipy.signal import gaussian to a set of points to smooth the curve in a rolling way.

def smooth_gaussian(data,window,std):
    g = gaussian(window,std,sym=True)
    con = np.convolve(g/g.sum(),data,mode='valid')
    con_shift = np.r_[np.full(int(window*0.5),np.nan),con,np.full(int(window*0.5),np.nan)]
    return con_shift

However, when doing this, the function offsets the values by std-1 positions from beggining and the end, filling it with nans.

smooth_gaussian(x, 5, 3)

x = [0.0,
 0.0185444808621514,
 0.04353375061397004,
 0.0,
 0.0,
 0.037009453627706586,
 -0.002574811929754943,
 -0.008514606925496021,
 -0.004525364338418344,
 0.01603344879624361,
 -0.008176977610078684,
 -0.01013245952744958,
 0.03996290965967278,
 0.004585506533521074,
 0.01120825808877464,
 -0.014854987339931331,
 0.027839959669720065,
 -0.01363659727385685,
 -0.009635505397613021,
 0.015952626842982574,
 0.0,
 -0.05723201212379214,
 -0.007780069509540553,
 0.004382874236289852,
 0.0,
 0.038106785772339016,
 0.0,
 0.0020824367482471916,
 0.009206907087771247,
 -0.005312729628236836,
 0.0003880320778995375,
 0.00336355496824854,
 0.0,
 0.0,
 -0.010513187775929366,
 -0.004263141180765177,
 4.0797273904902634e-05,
 0.0026873894750915357,
 0.04737815797347586,
 0.03626967474190855,
 -0.001326958533547213,
 0.006573790793200369,
 0.0,
 -0.012210432139001255,
 -0.005373057181818514,
 0.016644712998042103,
 -0.012265613043462619,
 0.0,
 0.11261448009585626,
 0.0,
 -0.012185874851013743,
 0.0005792221659147861,
 0.0,
 0.08345293709945158,
 0.0,
 -0.005246029136623375,
 0.0015148005522818237,
 0.001830695038247017,
 0.003687504607556841,
 0.0065507216003178395,
 0.00117405250448804,
 0.014311546898334035,
 0.017903880444956242,
 -0.006553897125751214,
 0.0,
 -0.07215579508846881,
 0.0,
 0.014181691708626792,
 0.017493159929742756,
 0.0,
 0.009550277653690674,
 0.0,
 -0.003356974394831732,
 0.0,
 0.0327208871241706,
 0.002795052678830963,
 -0.0037707485693199783,
 0.06211202416416171,
 0.0,
 0.015181750267714622,
 0.00334460827605957,
 -0.00021393411588840287,
 0.02503678667112154,
 0.0,
 0.0,
 -0.00466151928887153,
 0.06231556188921662,
 0.0,
 0.0,
 0.0,
 -0.11946455099766087,
 0.004232213476864264,
 -0.015338758071829633,
 0.0,
 -0.018906328574624865]

The result is:

array([            nan,             nan,             nan,  2.10349055e-02,
    2.35864806e-03,  1.73461936e-02,  1.59561400e-02, -4.26660062e-03,
   -5.73985110e-03,  4.92612899e-03,  3.27209493e-03, -6.97831701e-03,
    1.39369457e-02,  2.09944748e-02,  8.16964809e-03, -7.64435525e-04,
    6.03942428e-03,  5.97089401e-03, -9.67266366e-03,  2.57063885e-03,
    5.54861712e-03, -2.66962686e-02, -3.04662584e-02, -3.26920778e-03,
    2.94363128e-03,  1.80616142e-02,  1.79964811e-02,  2.36393111e-03,
    5.16194246e-03,  1.90536838e-03, -1.94961572e-03,  1.61117216e-03,
    1.59253745e-03, -2.09649704e-04, -5.07213168e-03, -6.95261269e-03,
   -2.21077837e-03,  2.54849096e-03,  2.46239791e-02,  3.94073221e-02,
    1.80239542e-02,  3.53458603e-03,  2.70115693e-03, -5.70966673e-03,
   -7.78881961e-03,  4.58850968e-03,  1.91504673e-03, -1.99075206e-03,
    5.26407851e-02,  5.26412919e-02, -2.42272310e-03, -5.44020155e-03,
    2.35800108e-03,  3.92900195e-02,  3.91217709e-02,  1.76058419e-05,
   -1.69258247e-03,  1.52960978e-03,  2.83255426e-03,  4.90808043e-03,
    4.16438980e-03,  8.00317308e-03,  1.50047039e-02,  5.75233874e-03,
   -4.67031944e-03, -3.41464738e-02, -3.35421047e-02,  5.07473061e-03,
    1.48998963e-02,  8.92716005e-03,  5.00770626e-03,  4.39834755e-03,
   -1.29671612e-03, -6.21006659e-04,  1.53823996e-02,  1.66109984e-02,
    2.31579756e-03,  2.75440728e-02,  2.95660756e-02,  9.05984198e-03,
    8.72256686e-03,  2.65024164e-03,  1.17818429e-02,  1.17765298e-02,
    6.03242779e-04, -3.67528126e-04,  2.71334365e-02,  2.91908596e-02,
    1.80942572e-03, -3.48822318e-03, -5.61009027e-02, -5.46800643e-02,
   -8.72490815e-03,             nan,             nan,             nan])

I don't want to replace the values with zeros but with the correct values that are part of the gaussian curve - normal distribution. Is it possible to do this?

Kind regards

user_4117624 · Answer

Try np.convolve with mode='same'

from scipy.signal import gaussian

def smooth_gaussian(data,window,std): 
    g = gaussian(window, std, sym=True) 
    con = np.convolve(g/g.sum(), data, mode='same') 
    return con

Make test data (s0, blue), add noise (s1, orange), and smooth it (con, green):

s0 = np.zeros(1000)
s0[:500] = 1
s1 = s0 + np.random.randn(len(s0))*0.1
con = smooth_gaussian(s,5,3) 
plt.plot(s0)
plt.plot(s1)
plt.plot(con)

And no nan:

In [71]: con[:10]                                                               
Out[71]: 
array([0.64101286, 0.85269195, 1.03414318, 1.05930853, 1.08433718,
       1.08190929, 1.08155865, 1.04812688, 1.02173964, 0.99050452])

In [72]: con[-10:]                                                              
Out[72]: 
array([-0.0148936 , -0.01155961, -0.03126125,  0.0143855 ,  0.03601306,
        0.05637481,  0.05888604,  0.064001  ,  0.04089501,  0.0207726 ])

Fill nan values after applying gaussian function

1 Answers