Scale MKAnnotationView with Map Zoom

I want to scale an MKAnnotationView as the map zooms in and out so that if my view takes up 100 meters, it'll continue to visually take up that much.

Here's what I have so far:

- (void)detectPinch:(UIPinchGestureRecognizer*)pinch{
    NSArray* annotations = self.mapView.annotations;
    for(id<MKAnnotation>annotation in annotations){
        if(![annotation isKindOfClass:[MKUserLocation class]] && [annotation isMemberOfClass:[CustomAnnotation class]]){
            CustomAnnotationView* annotationView = (CustomAnnotationView*)[self.mapView viewForAnnotation:annotation];
            CGAffineTransform transform;
            MKZoomScale currentZoomScale = self.mapView.bounds.size.width / self.mapView.visibleMapRect.size.width;
            self.previousMapScale = (self.previousMapScale == 0 ? currentZoomScale : self.previousMapScale);
            if(self.previousMapScale == currentZoomScale) return;
            else transform = CGAffineTransformMakeScale(pinch.scale*((currentZoomScale-self.previousMapScale)+1), pinch.scale*((currentZoomScale-self.previousMapScale)+1));
            annotationView.transform = transform;
            self.previousMapScale = (pinch.state == UIGestureRecognizerStateEnded ? 0 : currentZoomScale);
        }
    }
}

This almost works except for every pinch after the first one, the annotation view doesn't keep it's previous state and gets scaled weirdly. After the first call to the pinch gesture recognizer handler, it scales correctly for the rest of the gesture. It's just that initial part of the pinch.

Anybody know where I'm going wrong with my math?