Receiving audio from discord and covering Opus packets

I am having some trouble and would like some help.

I've been trying to make a discord bot that listens in to a user in voice channel and transcribe what the user is saying.

So far I have decided to use AssemblyAI API in order to transcribe the audio data I have extracted from discord.

AssemblyAI requires PCM 16 bit ,Mono channel audio data encoded in base64.

From Discord I receive Opus packets, with the option to encode it to base 64 ( but decided not to use it until the end)

Heres the code I have so far receiving from discord, obviously with research I referenced a lot of code that I found online and edited it for my use as well.

connect.receiver.speaking.on('start', async (user) => {
                    if (user.bot) return;
                    console.log(`Listening to <@${user}>`);
                    const stream = connect.receiver.subscribe(user, {
                        end: {
                            behavior: EndBehaviorType.AfterSilence,
                            duration: 1000
                        }
                    });
                    let buffer = [];
                    const encoder = new OpusEncoder(48000, 2);
                    stream.on('data', (data) => {
                        buffer.push(data);
                        
                        
                    });

                    stream.on('end', async () => {
                        buffer1 = Buffer.concat(buffer);
                        const duration = buffer1.length / 48000 / 4;
                        console.log("duration: " + duration);
                        if (duration > 1.0){
                            buffer = Buffer.concat(buffer);
                            if (socket){
                                pcm_buffer = encoder.decode(buffer);
                                mono_pcm_buffer = convert_audio(pcm_buffer);
                                base_64_mono_pcm_buffer= mono_pcm_buffer.toString('base64')
                                socket.send(JSON.stringify({ audio_data: base_64_mono_pcm_buffer}));
                            }
                            
                        }
                        async function convert_audio(input) {
                            try {
                                // stereo to mono channel
                                const data = new Int16Array(input)
                                const ndata = data.filter((el, idx) => idx % 2);
                                return Buffer.from(ndata);
                            } catch (e) {
                                console.log(e)
                                console.log('convert_audio: ' + e)
                                throw e;
                            }
                        }
                        
                        
                        
                        
                    });

                    /*
                    if (socket) {
                        socket.send(JSON.stringify({ audio_data: rawAudio }));
                    }
                    */
                });
}

I got the code working, and API working properly socket setup is working properly, however it seems like assemblyAI is not liking the data, meaning that there's something wrong with the conversion of audio data, or Im receiving it wrong. AssemblyAI is responding that they have received an audio file to transcribe, however no words are transcribed.

  [Symbol(kType)]: 'message',
  [Symbol(kData)]: '{"audio_start": 120, "audio_end": 150, "confidence": 0.0, "text": "", "words": [], "created": "2022-09-22T18:43:26.073654", "message_type": "FinalTranscript", "punctuated": true, "text_formatted": true}'
}

Obviously I talked in to the mic so it shouldn't be like this.

Could someone help me out in figuring out where I went wrong with the conversion or point me to the right direction in figuring out what to do to make it work ?