JaccardIndex(x, x) is not 1

user_10500331 0
Viewed 39

My understanding is that the JaccardIndex of a tensor with itself should turn 1 considering the intersection and union of a set with itself is always the set itself.

However when I experiment with JaccardIndex class from torchmetrics library I see the following.

from torchmetrics import JaccardIndex
import torch

pred = torch.tensor([1, 2, 19, 17, 17])
target = torch.tensor([1, 2, 3, 17, 4])

jaccard = JaccardIndex(num_classes=21)

jaccard(pred, pred)
Out[13]: tensor(0.1905)

jaccard(target, pred)
Out[14]: tensor(0.1190)

So instead of 1 the similarity of pred to itself is 0.1905.

Why is this so? What am I missing?

torch metrics
1 Answers

The metric is averaging the jaccard score over the 21 classes you defined. If you pass average=None to the metric it will show the jaccard score per class

from torchmetrics import JaccardIndex
import torch

pred = torch.tensor([1, 2, 19, 17, 17])
target = torch.tensor([1, 2, 3, 17, 4])

jaccard = JaccardIndex(num_classes=21)

jaccard(pred, pred)
Out: tensor([0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1.,
        0., 1., 0.])

Source: Docs

user_14907210
Related