Annotation id (groundtruth) problems in given 289 crop dataset

[ieellee]

I have statistic the given 289 crop datasets, and total categories id is
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 33, 35, 36, 37, 38, 44, 47, 48, 49, 50, 62, 63], but required predicted categories is [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 28, 30, 35, 36, 37, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 60, 64, 65].

It is obviously that categories 40, 41, 42, 43, 46, 51, 52, 53, 54, 55, 60, 64, 65 are missing in the given groundtruth, categories 25, 27, 29, 32, 33, 38, 62, 63 are not used in the required predict results. This may be a very serious problem in the following pipeline, although there exists included_class_id, but still missing some categories (e.g. 31, 38, 39...).

Another question is that what's id 0 means, and if I enconter class 2, should I make it to 60 or just keep 2 (60 includes 2).

This competition has taught me a lot of new knowledge, and I would like to thank the organizers @rhoadesScholar for promptly addressing my questions!😄😄😄

[rhoadesScholar]

Thank you @FrankieMAN-YU!

You are raising good questions. Crops have been exhaustively checked for accuracy, and though bugs can always squeeze through, this is not one of those cases. All is correct. Let me see if I can explain better than in the existing docs.

First, not all crops have all classes annotated. This is a practical matter in terms of annotating the amount of data we do. The training pipelines in this repo take this into account and do not calculate loss on classes without ground truth data in a given image.

There are classes that are annotated in the ground truth crops that we are not evaluating participants on - generally because we decided they were too hard or we did not have enough ground truth data for participants to have a fair shot of good performance.

0 always corresponds to background.

The all array does not necessarily include all of the classes labeled in a crop, because some class ID's are overlapping. For instance, lipid droplets (ld, 44) include both their corresponding lumens (ld_lum, 15) and their membranes (ld_mem, 4). Thus, a 44 id corresponds to either 15 or 14. Seeing 15 means that voxel is always also 44, but seeing 44 does not always mean the voxel is also 15. I strongly recommend you consider utilizing the visualization feature supplied in this repo, instead of viewing the all images exclusively. You will see that if all atomic classes corresponding to a group class are present, then that group class is also present - these labels are just separated into distinct arrays (e.g. data/jrc_hela-2/jrc_hela-2.zarr/recon-1/labels/groundtruth/crop1/ld and /nrs/cellmap/rhoadesj/cellmap-segmentation-challenge/data/jrc_hela-2/jrc_hela-2.zarr/recon-1/labels/groundtruth/crop1/ld_lum).

Hopefully this helps!

[ieellee]

Really helpful!!! Thanks a lot!