I'm attempting to receive a features vector of short wav (audio) files using wav2vec by using Hugging Face Transformers.
However, for unknown reasons, no matter which approach I use to control the output size, the results do not meet my requirements.
Ideally, I'd like to get all of the vectors to be the same length (e.g. 60K). I try to get it with the following command:
feature_extractor(input_audio, sampling_rate=16000, return_tensors="np", padding="max_length",
max_length=60000).input_values
That command helped me create a minimal boundary of the data size by padding all the vectors into a minimum of 60K length, but I was surprised to see vectors with 120K values created as well.
Then I remove the padding parameter in the hope of obtaining vectors with no padding but an upper boundary of 60K.
Based on the max_length documentation:
Maximum length of the returned list and optionally padding length
So I executed this line:
feature_extractor(input_audio, sampling_rate=16000, return_tensors="np",
max_length=60000).input_values
Unexpectedly, I receive vectors ranging in length from 20K to 120K. Not limited at all.
To reproduce my bug and results, I've included a snippet of code and a link to relevant audio data.
import librosa
import numpy as np
from transformers import Wav2Vec2FeatureExtractor
from pathlib import Path
p = Path(dataset_path)
audio_files = [i.parents[0] / i.name for i in p.glob('**/*.wav')]
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained('facebook/wav2vec2-base-960h')
for file in (audio_files):
input_audio, _ = librosa.load(file,
sr=16000)
features_with_padding = feature_extractor(input_audio, sampling_rate=16000,
return_tensors="np", padding="max_length", max_length=60000).input_values
features_without_padding = feature_extractor(input_audio, sampling_rate=16000,
return_tensors="np", max_length=60000).input_values
print(features_with_padding.shape, features_without_padding.shape)
In this drive folder, I attached 2 wav files that create about 80K length vector.
How could I create a one-size feature vector with a wav2vec transformer?