How can I give some hint phrases to OpenAI's Whisper ASR?

Franck Dernoncourt 0
Viewed 36

I use OpenAI's Whisper python lib for speech recognition. How can I give some hint phrases, as it can be done with some other ASR such as Google?

To transcribe with OpenAI's Whisper (tested on Ubuntu 20.04 x64 LTS with an Nvidia GeForce RTX 3090):

conda create -y --name whisperpy39 python==3.9
conda activate whisperpy39
pip install git+https://github.com/openai/whisper.git 
sudo apt update && sudo apt install ffmpeg
whisper recording.wav
whisper recording.wav --model large

If using an Nvidia GeForce RTX 3090, add the following after conda activate whisperpy39:

pip install -f https://download.pytorch.org/whl/torch_stable.html
conda install pytorch==1.10.1 torchvision torchaudio cudatoolkit=11.0 -c pytorch
hint-phrases openai-whisper openai speech-recognition python
1 Answers

2 potential places for hint phrases / boost:

I don't know how efficient it'd be. Also, another issue arises when the hint word is not in the dictionary, in which case one would need to add the hint word in the dictionary.

As Whisper is likely to attract many developers, hopefully some fork or pull request will have some options for hint phrases.

Franck Dernoncourt0
Related