Compare ground truth list of colours to another list of colours

user_972202 0
Viewed 116

I have 2 arrays/lists of colours. One represents the actual dominant colours in an image, the other represents a list colours an algorithm thinks are the dominant colours in an image.

I want to compare the 2 lists to see how *close the algorithm was to the actual ground truth list of colours. *2 colours are considered close (the same) if the euclidean distance between them is <= 25.

The problem is that the 2 lists will not necessarily be the same length. And the colours in each list wont be in any order. The colours will always be in the colour space CieLAB (cv2.COLOR_BGR2LAB).

Example of 2 lists I need to compare for similarity. Note they are in different order and different list lengths. But these 2 are considered the same because it found all the colours in the ground truth (plus some extra) and all those colours are <= 25 distance away.

ground_truth = [[76, 177, 34], [36, 28, 237], [204, 72, 63], [0, 242, 255]]
results = [[35, 29, 234], [200, 72, 63], [70, 177, 34], [0, 242, 250], [45,29,67], [3,90,52]]

I have built a validator below but I'm not sure if its correct? Any advice on how to implement my objective above? 

def validator(algoTuner, result, ground_truth):
    # Score = How close the colours in result are to the colours in ground_truth

    # p = pairwise distance between result and ground_truth
    # hits = get all distances from p that are <= max_dist
    # score = float(len(hits)) / len(ground_truth)
    dists = cdist(result, ground_truth, 'euclidean')
    score = float(len(dists[ dists < 25 ])) / len(ground_truth)
    return score

Edit after @Nakor's answer, would this be more correct? Remember the algorithm can find more colours than are in the ground truth. All that matters is the algorithm finds all the right colours in ground truth, any extra doesn't affect the score.

def validator(algoTuner, result, ground_truth, debug=False):
    dists = cdist(result, ground_truth, 'euclidean')
    correct_guesses = np.sum(dists<25, axis=1)
    correct_guesses = correct_guesses[ correct_guesses > 0 ]
    # score = correct_guesses.mean()
    score = float(correct_guesses.size) / len(ground_truth)

    if debug:
        print(len(correct_guesses))
        print(correct_guesses)
        print(score)
    return score
scipy opencv numpy python
1 Answers

I think the part where you calculate the score is not correct. Your counting the number of elements below 25 globally. But if I understood correctly, you're looking if, for each color in ground_truth there is at least one color in result that is less than 25 points away.

If this is the case, then I would modify your validator with:

def validator(algoTuner, result, ground_truth):
    # Score = How close the colours in result are to the colours in ground_truth

    # p = pairwise distance between result and ground_truth
    # hits = get all distances from p that are <= max_dist
    # score = float(len(hits)) / len(ground_truth)
    dists = cdist(result, ground_truth, 'euclidean')
    correct_guesses = np.sum(dists<25,axis=0)
    score = (correct_guesses>0).mean()
    return score

It returns the proportions of colors in ground_truth that are also present in result. In your example, the score would be 1.

user_62656300
Related