Why is stft(istft(x)) ≠ x?

Question

Using PyTorch, I have computed the short-time Fourier transform of the inverse short-time Fourier transform of a tensor. I have done so as shown below given a tensor x. For x, the real and imaginary part are equal, or the imaginary part is set to zero -- both produces the same problem.

torch.stft(torchaudio.functional.istft(x, n_fft), n_fft)

As shown in the image, only a single one of the stripes in the tensor remains after applying stft(istft(x)) -- all other stripes vanish. If stft(istft(x)) (bottom) was equal to x (top), both images would look similar. Why are they so different? It seems like stft(istft(x)) can pick up only certain frequencies of x.

x (top) and stft of istft of x (bottom)

I have also tried the same with scipy.signal.istft and scipy.signal.stft which causes the same issue. Moreover, I have tried it with a wide range of tensors x, e.g., different randomized distributions, images, and other stripes. Also, I have tried a variety of hyper-parameters for stft/istft. Only for x generated by a short-time Fourier transform from a sound wave, it works.

user_5483526 · Answer

A short-time Fourier transform produces a more data than there is in the original signal. Where a signal has N real samples, then STFT might have 4N complex samples -- 8 times more data.

It follows that the ISTFT operation must discard 7/8 of the data you provide it.

Most of the data in a STFT is redundant, and if you just make up values for all of the data, it is unlikely to correspond to a real signal.

In that case, an implementation of ISTFT will probably use a least-squares fit or other method of producing a signal with an STFT that matches your data as closely as possible, but it won't always be close.

Why is stft(istft(x)) ≠ x?

1 Answers