I need to implement continuous real-time speech to text that can use WebRTC as an audio source. I would love to use the speech_recognition library (here) because it has this wonderful .listen() method that works perfectly as a VAD and also makes it easy to create a wav file later to feed it to my STT model of choice. In short, I would like to structure it like this (credits to Ashutosh Dongare):
audio_input_file = "micAudioInput.wav"
audio_output_file = "ttsAudioOutput.wav"
while True:
try:
with sr.Microphone() as SR_AudioSource:
print("Say something...")
mic_audio = r_audio.listen(SR_AudioSource,1,4) # timeout=1, phrase_time_limit=4
except: #This is required if there is no Audio input or some error
print("Found not capture mic input or no audio...")
continue #continue next mic input loop if there was any error
with open(audio_input_file, "wb") as file:
file.write(mic_audio.get_wav_data())
file.flush()
file.close()
# Check if there is input audio file saved otherwise continue listening
if not os.path.exists(audio_input_file):
print("no input file exists")
continue
# Read audio file into batches and create input pipelie for STT
batches = split_into_batches(glob(audio_input_file), batch_size=10)
readbatchout = read_batch(batches[0])
input = prepare_model_input(read_batch(batches[0]), device=device)
#feed to STT model and get the text output
output = stt_model(input)
you_said = decoder(output[0].cpu())
print(you_said)
if(you_said == ""):
print("No speech recognized...")
continue
#check if user wants to stop
if(re.search("exit",you_said) or re.search("stop",you_said) or re.search("quit",you_said)):
break
It works perfectly using a physical microphone locally. The only issue is that I am pretty sure that speech_recognition only accepts microphones of the device it is run on as an AudioSource (and I know that PyAudio only works that way) as the documentation states that the Microphone class
[c]reates a new Microphone instance, which represents a physical microphone on the computer.
Does this mean it is not possible to use it with WebRTC as source? And if not, what would be a good substitute to its .listen()? Also: Is this concept of an infinite loop generally even feasible in the browser or is it a bad idea? I plan on implementing this in Django and handling the WebSocket
server with Django Channels.