improving the detection accuracy by using euclidean distance between its Centroids

Question

I'm working on a project where i have to detect colored cars form video frames taken from Bird's eye view.

For detection i used Histogram backprojection to obtain a binary image that suppose to contain only the target region of interest.

The process works fine until i tried to generalize the detection by testing it on video that contains object with similar color distribution ( like me crawling under the table and parts of my T-shirts are visible)

As you can see, both car and irrelevant objects are moving and the detection results are:

As you see irrelevant objects that share similar color distribution are shown in the binary images. However, thanks to Stack overflow experts i could improve the detection by telling the algorithm to choose the blob that represents the target object by adding the following constraints:

1-Rectangularity check 2-area and ratio check

with above constraints i could get rid of large irrelevant objects that are detected. However, for small object (see binary images), it doesn't work that much as the rectangularity for the target object (small red car) ranges between (0.72 and 1) and the small irrelevant objects do fall in this range. So i decided to add another constraint which calculating the distance between centroids of the car moving every 5 successive frames and threshold depending on that distance by doing the flowing:

 import scipy.spatial.distance
 from collections import  deque

 #defining the Centroid
 centroids=deque(maxlen=40) #double ended queue containing the detected centroids
 .
 .
 .
 centroids.appendleft(center)
 #center comes from detection process. e.g centroids=[(120,130), (125,132),...
 Distance = scipy.spatial.distance.euclidean(pts1_red[0], pts1_red[5])                        
 if D<=50:
    #choose that blob

So tested that on different videos and turns out the the distance between the centroids ranges between 0 and 50 (0 when the car stops).

So my question is:

Is there a way i can invest this property so that it helps enhancing the detection in such a way so that the detection ignores the T-shirt?, Since when the car is no longer visible and the irrelevant object stays it will calculate the distance difference of the irrelevant object and this distance will get small till it is less than 50!

Thanks in Advance

user_8505509 · Answer

Based on the information provided by the OP, here is an approach to solve this problem.

Sample Images

I have create a few sample images that roughly represent the objects moving across time. The center object represents the car that we are seeking and the other objects have been detected incorrectly by the classifier.

The first four images represent a car ( centre object )moving from left to right, along with two other objects that have been detected incorrectly. In the fifth image, the car has moved out of the frame but two incorrect detections are still present. The sixth frame consists of a new car entering into the frame with other incorrect detections.

Solution - Code

The comments contain information regarding the algorithm. We are computing the centroid of each blob and comparing it with the centroid of the previously detected/extracted blob.

import os
import cv2
import numpy as np
# Reading files and sorting them in the right order
all_files = os.listdir(".")
all_images = [file_name for file_name in all_files if file_name.endswith(".png")]
all_images.sort(key=lambda k: k.split(".")[0][-1])
print(all_images) # 

# Initially, no centroid information is available. 
previous_centroid_x = -1
previous_centroid_y = -1
 
DIST_THRESHOLD = 30
for i, image_name in enumerate(all_images):
    rgb_image = cv2.imread(image_name)
    height, width = rgb_image.shape[:2]
    gray_image = cv2.cvtColor(rgb_image, cv2.COLOR_BGR2GRAY)
    ret, thresh = cv2.threshold(gray_image, 127, 255, 0)
    contours, hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)  
    blankImage = np.zeros_like(rgb_image)
    for cnt in contours:
        # Refer to https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_contours/py_contour_features/py_contour_features.html#moments
        M = cv2.moments(cnt)
        cX = int(M["m10"] / M["m00"])
        cY = int(M["m01"] / M["m00"])
        # Refer to https://www.pyimagesearch.com/2016/04/11/finding-extreme-points-in-contours-with-opencv/
        # https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_contours/py_contour_properties/py_contour_properties.html#contour-properties
        extLeft = tuple(cnt[cnt[:, :, 0].argmin()][0])
        extRight = tuple(cnt[cnt[:, :, 0].argmax()][0])
        extTop = tuple(cnt[cnt[:, :, 1].argmin()][0])
        extBot = tuple(cnt[cnt[:, :, 1].argmax()][0])
        color = (0, 0, 255)
        if i == 0: # First frame - Assuming that you can find the correct blob accurately in the first frame
            # Here, I am using a simple logic of checking if the blob is close to the centre of the image. 
            if abs(cY - (height / 2)) < DIST_THRESHOLD: # Check if the blob centre is close to the half the image's height
                previous_centroid_x = cX # Update variables for finding the next blob correctly
                previous_centroid_y = cY
                DIST_THRESHOLD = (extBot[1] - extTop[1]) / 2 # Update centre distance error with half the height of the blob
                color = (0, 255, 0) 
        else:
            if abs(cY - previous_centroid_y) < DIST_THRESHOLD: # Compare with previous centroid y and see if it lies within Distance threshold
                previous_centroid_x = cX
                previous_centroid_y = cY
                color = (0, 255, 0) 

        cv2.drawContours(blankImage, [cnt], 0, color, -1) 
        cv2.circle(blankImage, (cX, cY), 3, (255, 0, 0), -1)
        cv2.imwrite("result_" + image_name, blankImage)

Updating the threshold enables the algorithm to track the object's centroid across the frames. Since the object can move up and down a little, we want to match the centroids of objects found in the current frame, to the centroid of the car found in the previous frame.

Solution - Results

Green - Selected blob Red - Rejected blob Object centres have also been marked for reference.

Note - This is not a perfect solution. It has several limitations, but it can help you to design an approximate solution for your problem.

improving the detection accuracy by using euclidean distance between its Centroids

1 Answers