How to get letters position relative to audio time in huggingsound?

The HuggingSound creator here! The transcriptions returned by the transcribe method contain the start/end timestamps in milliseconds of the audios that you passed to the method...

from huggingsound import SpeechRecognitionModel

model = SpeechRecognitionModel("jonatasgrosman/wav2vec2-large-xlsr-53-english")
audio_paths = ["/path/to/sagan.mp3", "/path/to/asimov.wav"]

transcriptions = model.transcribe(audio_paths)

t = transcriptions[0] # getting the first audio transcription

print(t)

# {
#   "transcription": "extraordinary claims require extraordinary evidence", 
#   "start_timestamps": [100, 120, 140, 180, ...], <- milliseconds
#   "end_timestamps": [120, 140, 180, 200, ...], <- milliseconds
#   "probabilities": [0.95, 0.88, 0.9, 0.97, ...]
# }

The start_timestamps[i] gives you the time in milliseconds when the i-th letter was detected in the transcription. The end_timestamps[i] gives you the time in milliseconds when the i-th letter stopped being detected in the transcription.

So you can use the start_timestamps and end_timestamps lists to get the timing and even the duration of each letter of an audio file:

# What's the first letter of the audio file?
print(t["transcription"][0]) # e

# When's the detection of the first letter of the audio?
print(t["start_timestamps"][0]) # at 100ms
    
# How long is the first letter of the audio?
print(t["end_timestamps"][0] - t["start_timestamps"][0]) # (120-100) = 20ms long

... And to get the letter position given a time:

# What's the letter at 108ms?
for i, letter in enumerate(t["transcription"]):
    if t["start_timestamps"][i] <= 108 <= t["end_timestamps"][i]:
        print(letter) # e
        break