AVPlayer doesn't respond to commands from bluetooth device

My ios app uses AVQueuePlayer to play audio.

The play/pause controls work properly from Control Center and from ios lock screen, but not for commands invoked from bluetooth device. E.g., the callbacks in my app are never invoked when double-tapping airpods or other third-party bluetooth headphones.

I'm running on an iphone 8 plus device with iOS 15.

I'm using https://developer.apple.com/documentation/avfoundation/media_playback_and_selection/creating_a_basic_video_player_ios_and_tvos/controlling_background_audio.

On app startup, it sets up the audio session like this:

        let audioSession = AVAudioSession.sharedInstance()
        do {
            try audioSession.setCategory(.playAndRecord,
                                         mode: .spokenAudio,
                                         options: [
                                            .allowBluetooth,
                                            .defaultToSpeaker
                                         ])
            try audioSession.setActive(true)

The app listens for play/pause events like this.

        let commandCenter = MPRemoteCommandCenter.shared()
        commandCenter.playCommand.addTarget { event in
print("play")
    return .success
        }
        commandCenter.pauseCommand.addTarget { event in
print("pause")
    return .success
        }
commandCenter.togglePlayPauseCommand.addTarget { event in
    print("toggleplaypause")
    return .success
}

Playback works correctly through all bluetooth devices. On lock screen or control center, play/pause cause "play" and "pause" to get printed. But nothing is printed when tapping airpods. (Tapping airpods works fine in any other audio app.)

Any ideas what might be missing?