Routing audio output between multiple bluetooth devices with iOS using AVAudioSession

Got a video conference app, implementing device selection feature of switching between Earpiece, Speaker, and connected Bluetooth devices. All seems to function appropriately apart from switching between bluetooth devices themselves.

For some reason, only the last connected device gets the audio routed to it, and you can't switch back to the other ones even if they're available in availableInputs of AVAudioSession SharedInstance, using setPreferredInput with OverridePortNone

I tried searching for resolutions but found the same unanswered issue from 5 years ago, I tried doing the same as changing setActive calls but was also unsuccessful.

Following is the test code, which is taken from here:

    AVAudioSession *_audioSession = [AVAudioSession sharedInstance];

    AVAudioSessionCategoryOptions _incallAudioCategoryOptionsAll = AVAudioSessionCategoryOptionMixWithOthers|AVAudioSessionCategoryOptionAllowBluetooth|AVAudioSessionCategoryOptionAllowAirPlay;

    [_audioSession setCategory:AVAudioSessionCategoryPlayAndRecord
                   withOptions:_incallAudioCategoryOptionsAll
                   error:nil];

    [_audioSession setMode:AVAudioSessionModeVoiceChat error: nil];


    RCT_EXPORT_METHOD(setAudioDevice:(NSString *)device
                  resolve:(RCTPromiseResolveBlock)resolve
                  reject:(RCTPromiseRejectBlock)reject) {
    
    BOOL success;
    NSError *error = nil;
    
    NSLog(@"[setAudioDevice] - Attempting to set audiodevice as: %@", device);
    
    if ([device isEqualToString:kDeviceTypeSpeaker]) {
        success = [_audioSession overrideOutputAudioPort:AVAudioSessionPortOverrideSpeaker error:&error];
    } else {
        AVAudioSessionPortDescription *port = nil;

        for (AVAudioSessionPortDescription *portDesc in _audioSession.availableInputs) {
            if ([portDesc.UID isEqualToString:device]) {
                port = portDesc;
                break;
            }
        }

        if (port != nil) {
            [_audioSession overrideOutputAudioPort:AVAudioSessionPortOverrideNone error:nil];
            success = [_audioSession setPreferredInput:port error:&error];
            
            if(error != nil)
            {
                NSLog(@"setAudioDevice %@ %@", error.localizedDescription, error);
            }
        } else {
            success = NO;
            error = RCTErrorWithMessage(@"Could not find audio device");
        }
    }
    if (success) {
        resolve(@"setAudioDevice success!");
        NSLog(@"resolved success");
    } else {
        reject(@"setAudioDevice", error != nil ? error.localizedDescription : @"", error);
        NSLog(@"sent reject");
    }
 }

So how can I make it that we're able to successfully change from one bluetooth device to other?