How to tap on object from an image and track it from sequence of images using Vision and Core ML framework

I am developing an app using new Core ML framework. What I am trying to achieve is as follows: 1. Select a image and tap on any object from it to draw rectangle 2. After that track that object in multiple images just running in for loop

Currently I am doing with following process

1. Detect object when user tas and store it VNDetectedObjectObservation = VNDetectedObjectObservation(boundingBox: convertedRect)

2. Create VNTrackObjectRequest for VNImageRequestHandler to perform the request

But not getting proper result. Any help will be appreciated.