iOS SamplerUnit stuck notes

I'm developing an AUSampler based application for iOS12.4x (I'm on High Sierra so can't develop for iOS13+, I'm using Xcode 10.3).

It manages incoming MIDI messages through the MIDIReadProc handler.

I noticed stuck notes (as if some note off messages werent't handled) when playing many note repeatedly at the same time. I called MusicDeviceMIDIEvent directly from MIDIReadProc handler, however it didn't fix the problem.

UPDATE : using new AVAudioUnitSampler (and AVAudioEngine/AVAudioSession instead AUGraph) didn't helped.

UPDATE : Using AudioKit 4.8 and wrapper classes (AKMidiSampler/AKAppleSampler) for Apple's AVAudioUnitSampler fixed the problem, so the problem may come from the way I manage midi messages (however it is similar as how AudioKit does).

New code I'm using (older code with MIDIReadProc was Objective-C) :

err = MIDIInputPortCreateWithBlock(client, portName, &port, onMIDIMessageReceived)

And handler defined as :

func onMIDIMessageReceived(message: UnsafePointer<MIDIPacketList>,
srcConnRefCon: UnsafeMutableRawPointer?) {

    let packetList: MIDIPacketList = message.pointee
    let n = packetList.numPackets
    var packet = packetList.packet
    for _ in 0 ..< n {
      
        let mes: UInt8 = packet.data.0 & 0xF0
        let ch: UInt8 = packet.data.0 & 0x0F
        if mes == 0x90 && packet.data.2 != 0 {
           
            let noteNo = packet.data.1
            let velocity = packet.data.2
            DispatchQueue.main.async {
                self.delegate?.noteOn(ch: ch, note: noteNo, vel: velocity)
            }
        } else if (mes == 0x80 || mes == 0x90) {
            
            let noteNo = packet.data.1
            let velocity = packet.data.2
            DispatchQueue.main.async {
                self.delegate?.noteOff(ch: ch, note: noteNo, vel: velocity)
            }
        }
        let packetPtr = MIDIPacketNext(&packet)
        packet = packetPtr.pointee
    }
}

I also tested without DispatchQueue.main.async (using it instead in the delegate when calling the sampler methods), same problem.

I would like not having to rely on Audiokit as I think this is overhead for what I plan (I want to learn/control all of my code).