MKMapView refresh map after removing annotation

I have the following function in an extension of MKMapView that let me reload a specific annotation and keep it selected if needed and the issue is every time the self.removeAnnotation is called, the whole map is reloaded (or at least all the pins "jumped" as if a reload occurred)

Is there a way to achieve a reload of a single annotation without having the visual of a whole map reloading ?

 func reloadAnnotation(_ annotation: MKAnnotation, keepSelection: Bool = true) {
        let isSelected = selectedAnnotations.contains(where: annotation.isEqual(_:))
        //already tried **UIView.performWithoutAnimation** which decrease the jumping effect
        // UIView.performWithoutAnimation {
          self.removeAnnotation(annotation)
          self.addAnnotation(annotation)
        // }
        guard isSelected && keepSelection else {
            return
        }
        self.selectedAnnotations.append(annotation)
    }