record audio with exact timestamps

Viewed 847

I want to record raw audio on an android wear device, and exactly synchronize it with other sensor data, such as the accelerometer. To record raw audio, I use AudioRecord, but how can I determine the correct timestamp for any part of the audio stream?

The basic problem is that i get raw audio frames from the buffer, without information on how long those frames were in the buffer, how many frames were lost, etc.

So to get the frame data I call AudioRecord.read() in a loop, but how can I tell when a specific frame was recorded?

I can see three ways to determine a timestamp, and all of them seem wrong to me:

  1. Get the current system time before or after calling AudioRecord.startRecording(), by calling SystemClock.elapsedRealtimeNanos(), and then calculate the time of each chunk based on the sample rate and frame size. But according to my tests there is a significant delay, until the recording actually starts. The time would also get out of sync whenever some frames are lost, e.g. when processing is stalled for a moment.

  2. Get the current system time before or after AudioRecord.read(). But that is probably not the time that this data was recorded, that could have happened a bit earlier. And if no data is available yet, read() will block and the timestamp taken before will be totally wrong.

  3. AudioRecord.getTimestamp() gives the time of the frame that is currently recording. But that is not necessarily the frame that I am currently reading, I could be a couple frames behind, or I could have missed some frames. And according to initial tests, for some time after the start the timestamp is simply zero.

This would leave me with 6 probably wrong timestamps, shown in the code below. Is any of those even close to accurate?

recorder = new AudioRecord(...)

// timeBeforeStart =  SystemClock.elapsedRealtimeNanos()
recorder.startRecording()
// timeAfterStart =  SystemClock.elapsedRealtimeNanos()

...

// timeBeforeRead =  SystemClock.elapsedRealtimeNanos()
// recorder.getTimestamp(frameTimeBeforeRead, AudioTimestamp.TIMEBASE_BOOTTIME)
recorder.read(myArray, 0, mySize);
// recorder.getTimestamp(frameTimeAfterRead, AudioTimestamp.TIMEBASE_BOOTTIME)
// timeAfterRead =  SystemClock.elapsedRealtimeNanos()

Which of these would be the most accurate way to determine the timestamp corresponding to the data in the byte array?

Is there a different method for retrieving raw audio data with accurate timestamps?

1 Answers

You need to create a correlated audio and accelerometer signal - You might also throw flashlight, camera and speaker, etc. into the mix.

You need to test which actors & sensors have which delay and jitter, to find the optimal solution (without using an external device) to be able to correlate events in order to calculate the received audio delay and jitter.

Related