Unable to record audio with cpal and encode with opus. Always creates improper file

Viewed 385

I have been working for several hours trying to simply record audio from my microphone and encode it with opus so I can ultimately stream it and play it elsewhere. Currently, I am struggling with trying to get it to simply play a file. I based this pretty heavily based upon the cpal examples as well as the libopus c example. Currently, it just outputs a nonsense file that VLC cant even read. However, if I print the raw bytes as they are encoded, I can definitely tell something is happening. I have also tried messing with endianess, but it did not work at all. I have also been using rubato to resample the raw output into a way the opus can use.

fn main() -> Result<(), anyhow::Error> {

    let mut encoder = opus::Encoder::new(48000, opus::Channels::Stereo, opus::Application::Voip).unwrap();

    let mut resampler = rubato::FftFixedInOut::<f32>::new(44100, 48000, 896, 2);

    let host = cpal::default_host();

    let device = host.default_input_device().unwrap();


    println!("Input device: {}", device.name()?);

    let config = device
        .default_input_config()
        .expect("Failed to get default input config");
    println!("Default input config: {:?}", config);

    println!("Begin recording...");

    let err_fn = move |err| {
        eprintln!("an error occurred on stream: {}", err);
    };

    let sample_format = config.sample_format();

    // let socket = std::net::UdpSocket::bind("192.168.1.82:1337")?;
    const PATH: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/recorded.pcm");
    let socket = std::fs::File::create(PATH).unwrap();
    let mut socket = BufWriter::new(socket);

    let stream = device
        .build_input_stream_raw(
            &config.into(),
            sample_format,
            move |data, _: &_| write_input_data_f32(data, &mut encoder, &mut resampler, &mut socket),
            err_fn,
        )
        .unwrap();


    stream.play()?;

    std::thread::sleep(std::time::Duration::from_secs(10));
    drop(stream);

    Ok(())
}

type ResamplerHandle = rubato::FftFixedInOut<f32>;
// type SocketHandle = std::net::UdpSocket;
type SocketHandle = BufWriter<std::fs::File>;

fn write_input_data_f32(
    input: &Data,
    encoder: &mut Encoder,
    resampler: &mut ResamplerHandle,
    socket: &mut SocketHandle,
) {

        let mut inp = input.as_slice::<f32>().unwrap().to_vec();

        inp.truncate(resampler.nbr_frames_needed());
        if inp.len() < resampler.nbr_frames_needed() {
            inp.append(&mut vec![0f32; resampler.nbr_frames_needed() - inp.len()]);
        }
        let mut wave_out = resampler.process(&vec![Vec::from(inp); 2]).unwrap();//[0].to_owned();

        use itertools::interleave;
        let v1 = wave_out[0].to_owned();
        let v2 = wave_out[1].to_owned();
        let v = interleave(v1.chunks(1), v2.chunks(1)).flatten().copied().collect::<Vec<f32>>();

        let buff = encoder.encode_vec_float(v.as_slice(), 960).unwrap();

        use std::io::Write;
        socket.write(&buff);
}
2 Answers

It looks like you're writing raw audio frames to a file, which most likely is not what you are looking for. Most audio files aren't just raw data, they use an audio container with a header and other features. For Opus, you most likely want to use an Ogg container for simply saving audio to a file (as opposed to e.g. streaming).

The issue, I found, is a misuderstanding with how opus was working. It seems to work distinctly in chunks and not a stream of input. So I went ahead and started streaming them over the network and when I put in the exact output chunks it worked perfectly!

Related