[FEATURE]Native Support for End-to-End Panoptic Segmentation (based on Cellpose-SAM) #1388
Description
I am writing to follow up on the discussion in Issue 1287(#1287) where it was clarified that the semantic classification logic currently resides in paper/cpsam/semantic.py as a proof of principle.
While Cellpose-SAM has demonstrated the capability to perform Panoptic Segmentation (simultaneous instance segmentation and semantic classification), this functionality is currently decoupled from the main package. Users like myself currently have to rely on fragmented workflows to achieve this.
Proposal I propose transforming the existing proof-of-concept into a first-class feature within the Cellpose ecosystem. Specifically, I am interested in developing a native module that supports End-to-End Panoptic Segmentation. This would allow the model to output both:
Instance Masks (Flows)
Semantic Classes (Classification Head)
Action Plan I have already reached out to [email protected] via email with a detailed proposal regarding potential collaboration and implementation details. My goal is to implement this feature with full support for both CLI and GUI, ensuring it remains backward compatible while extending Cellpose to be a universal platform for "Pixel-to-Phenotype" analysis.
I look forward to your thoughts and any architectural guidance you might have.
Best regards