What would be the ideal approach to model a CNN?

user_8048773 0
Viewed 45

I am trying to perform detection of a certain type of sound in audio files. These audio recordings have variable lengths and the type of sound that I want to detect is usually around 1~5 seconds long and I have the labels of the dataset (onset and offset of when events happen).

My initial approach was by treating it as a binary classification problem. Where I compute the mel spectrogram each half a second (for example). I would label that spectrogram with a 0 if there wasn't a event in those 0.5s and labeled it 1 if the other way.

In what way could I fight this? I am trying to change by passing 0.1 instead of 1 (assuming the previous example). Basically labeling the percentage of the the event happening in the image: labels [0~1] instead of {0,1}.

Many thanks.

conv-neural-network deep-learning classification regression
1 Answers

I have approached problems like this by using a fixed input-size CNN to do a simple classification and then called the CNN multiple times as you scan across your variable length sample (1-5 sec sound bite).

For example, let's say you create a CNN that inputs 0.2s of data, the input size is now fixed. You can compute a {0, 1} label for that 0.2s based on whether the center point of the sample is within an event as you defined in your question. You could try different input sizes using the same method.

Now you ask the CNN to make a prediction at every point in your 1-5 second sample. To start with you pass the CNN the first 0.2s of data, then step forward one or more data points (your step size is a hyper-parameter you can tune). Let's say your step size is 0.1s, your second step would produce a CNN classification using the data from 0.1s to 0.3s in your sample. Continue until you reach the end of your sample. You now have classifications across the sample. In principle you could get a classification at every data point so you have as many predictions as you have data points. A rolling median filter (see pandas) is a great way to smooth out the predictions.

This is a very simple CNN to set up. You also benefit by increasing your training data quite a bit because each sound file is now many training samples. Your resolution for predictions is very granular with this method.

Here's a paper that describes the approach in greater depth (there's also a slightly earlier version on arXiv by the same title if that's pay walled for you), start reading at Section 3 onward:

https://academic.oup.com/mnras/article/476/1/1151/4828364

In that paper we're working with 1D astronomy data, which is structured basically the same as 1D audio data, so the technique will apply. In that paper I'm doing a bit more than just classification, using the same technique I'm localizing zero or more events as well as characterizing those events (I would start with just the classification for your purposes). So you can see that this approach extends quite well. In fact even multiple events that partially overlap each other in time can be identified and extracted effectively.

user_47908710
Related