Is 'ignore_index=0' necessary for binary classification?

[dakomura]

I would like to use the IoU metric for the binary segmentation task (background or target).
The prediction matrix has only one channel for the target class (not 2 channels for background and target class).

The current code is as follows.

self.train_metric = torchmetrics.IoU(
    num_classes=2,
    dist_sync_on_step=True)

Do I need to add 'ignore_index=0' when I'm only interested in the target class?

[SkafteNicki]

Hi @dakomura,
Yes, you will need to set ignore_index=0 in your usecase

[dakomura]

Thank you!!

[dakomura]

Hi @SkafteNicki

I added ignore_index=0 but the behavior was strange.

So I checked the value of confmat in the code (_iou_from_confmat) and found that the value was like this.

tensor([[1.9591e+05, 7.1420e+06],
        [1.8340e+03, 3.3070e+04]], device='cuda:0')

I think the IoU for the target class can be calculated as follows:

confmat[1,1] / (confmat[0,1] + confmat[1,0] + confmat[1,1]) = 0.0046

When I added ignore_index=0, confmat became

tensor([[0, 0],
        [1.8340e+03, 3.3070e+04]], device='cuda:0')

, where the essential information (confmat[0,1]) was lost.

When I didn't add ignore_index=0, score was

[0.0267, 0.0046]

, where the 2nd component is the IoU for the target class.

So I think the following code should be removed in my case.

    # Remove the ignored class index from the scores.
    if ignore_index is not None and 0 <= ignore_index < num_classes:
        confmat[ignore_index] = 0.0

Am I right?

[geografif]

Have you resolved the issue? What was your conclusion? @dakomura

[Borda]

@geografif to close the loop here — the original advice to use ignore_index=0 was incorrect for this use case, and @dakomura's confusion was justified. Here's the clarification:

You do NOT need ignore_index=0 for standard binary segmentation.

ignore_index is meant to exclude a class that shouldn't participate in the metric computation at all (e.g., an "unlabeled" or "void" class in semantic segmentation). If your class 0 is a real class (background), ignoring it would discard meaningful confusion matrix entries.

The correct setup for binary segmentation (v1.9.0):

from torchmetrics.classification import BinaryJaccardIndex

# preds: (N, H, W) — 0 or 1
# target: (N, H, W) — 0 or 1
iou = BinaryJaccardIndex()
score = iou(preds, target)

No ignore_index needed. If you want per-class IoU (background + foreground separately):

from torchmetrics.classification import MulticlassJaccardIndex

iou = MulticlassJaccardIndex(num_classes=2, average="none")
per_class = iou(preds, target)  # tensor([iou_bg, iou_fg])

When you WOULD use ignore_index: if your labels include a value like 255 or -1 meaning "don't care / unlabeled" — then ignore_index=255 makes sense to exclude those pixels.

Docs: JaccardIndex