I'm testing the MFCC feature from tensorflow.signal implementation. According to the example (https://www.tensorflow.org/api_docs/python/tf/signal/mfccs_from_log_mel_spectrograms), it is computing all 80 mfccs and then taking the first 13.
I have tried both above and "compute first 13 directly" approach and the result is very different:
All 80 first, then take first 13:
Compute first 13 directly:
Why the big difference, and which one should I use if I'm passing this as feature to CNN or RNN?
That's because of the nature of MFCC. Remember that these coefficients are calculated over the frequency range on the mel scale that you provide via lower_edge_hertz and upper_edge_hertz in the linked code.
What it means in practice:
"Calculate 13 coefficients directly": take frequency range [80.0, 7600.0] and divide it into 13 bins. Eventually, you will get 13 coefficients that reflect amplitudes of the corresponding spectrum (see MFCC algorithm)
"All 80 first, then take first 13": take frequency range [80.0, 7600.0] and divide it into 80 bins. Now, take only first 13 coefficients. In practice, that means that you're looking into much narrow and fine grained spectrum, in this case roughly in the human speech frequency range [80, 400] Hz (roughly speaking, back of the envelope calculations). Makes sense if you're into human speech recognition, as you can focus on more subtle variations while ignoring the higher frequency spectrum (that is less interesting from our audiory system perspective).
