Writing and reading audio from a tempfile.TemporaryFile

I want to convert an mp3 audio file to a wav file "on the fly":

import pydub
audio_segment = pydub.AudioSegment.from_mp3(audio_path)

This works:

with open("temp.wav", mode="wb") as temp_file:
    audio_segment.export(out_f=temp_file, format="wav")
with open("temp.wav", mode="rb") as temp_file:
    signal_wave = wave.open(temp_file)
    sample_rate = 16000
    signal = numpy.frombuffer(
        signal_wave.readframes(sample_rate), 
        dtype=numpy.int16
    )

I thought the tempfile module would give me a more elegant way, where I don't need clean up any file. However, there seems to be a problem writing to and then reading from the TemporaryFile object:

with tempfile.TemporaryFile(mode="wb") as temp_file:
    audio_segment.export(out_f=temp_file, format="wav")
    signal_wave = wave.open(temp_file)
    sample_rate = 16000
    signal = numpy.frombuffer(
        signal_wave.readframes(sample_rate), 
        dtype=numpy.int16
    )

Error message:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-38-785440e73545> in <module>
      4     sample_rate = 16000
      5     signal = numpy.frombuffer(
----> 6         signal_wave.readframes(sample_rate),
      7         dtype=numpy.int16
      8     )

AttributeError: 'Wave_write' object has no attribute 'readframes'