Tutorial Proposal: Visual Medical Imaging Workflows with ComfyUI Integration for Clinical Lesion Detection and Segmentation

[kumabear618]

I propose creating a comprehensive tutorial that integrates MONAI with ComfyUI's node-based visual workflow system, specifically targeting surgical, radiological, and other consultants seeking to perform AI-assisted lesion detection and segmentation without writing code.

This tutorial will demonstrate a complete clinical workflow:
CT scan review → AI-suggested lesions → radiologist refinement → quantitative analysis → structured report generation, with extensive visualization at every step.

Problem Statement
Current MONAI tutorials are excellent for researchers and ML engineers but require Python programming knowledge. Clinical users who could benefit most from MONAI's powerful medical imaging capabilities often lack coding experience, creating a barrier to adoption in real-world clinical settings.

Proposed Solution
A visual, node-based interface using ComfyUI that allows clinicians to:

• Build medical imaging workflows by connecting visual nodes (no coding required)
• Leverage pre-trained MONAI models through drag-and-drop
• Interactively refine AI predictions
• Generate clinical reports with quantitative measurements
• Visualize results in 2D/3D at every pipeline stage

Target Audience
Primary: Licensed clinicians, radiologists, and medical technicians
Secondary: Medical students learning medical imaging AI
Tertiary: Researchers prototyping clinical workflows

Proposed Tutorial Content
Title: Visual Medical Imaging Workflows: ComfyUI Custom Nodes for Clinical Liver Lesion Segmentation

Clinical Workflow (Liver Lesion Segmentation Use Case)

1. DATA INGESTION & PREPROCESSING
   —> Load DICOM/NIfTI abdominal CT scan
   —> Visualize: Multi-planar reconstruction (Axial/Coronal/Sagittal)
   —> Metadata extraction (patient info, scan parameters)
   —> Intensity windowing (liver window settings)
   —> Quality control checks

2. AI-ASSISTED LESION DETECTION
   —> RetinaNet-based detection model for liver lesions
   —> Visualize: 3D bounding boxes overlaid on CT
   —> Confidence scores per detection
   —> Lesion characteristics (volume, location, HU density)
   —> False positive filtering

3. LESION SEGMENTATION
   —> UNet/SegResNet for precise lesion boundaries
   —> Visualize: 3D mask overlay with transparency
   —> Multi-class segmentation (cyst, hemangioma, metastasis, HCC)
   —> Morphological refinement
   —> Uncertainty estimation

4. RADIOLOGIST REFINEMENT
   —> Interactive editing (DeepEdit-style)
   —> Add/remove lesions manually
   —> Adjust segmentation boundaries
   —> Annotate findings
   —> Real-time AI re-inference

5. QUANTITATIVE ANALYSIS
   —> Lesion volume calculation (mm³)
   —> Radiomics features extraction
   —> Growth tracking (if prior scans available)
   —> RECIST measurements
   —> Statistical summaries

6. REPORT GENERATION & EXPORT
   —> Structured report (JSON/XML)
   —> Visual summary (key slices with annotations)
   —> 3D rendering export
   —> DICOM SR (Structured Report)
   —> Integration-ready formats (PACS/RIS)

Technical Implementation (tentative)

Proposed Custom ComfyUI Node Categories (tentative):

1. Medical Image I/O Nodes:
   LoadMedicalImage - DICOM/NIfTI loader with metadata
   SaveMedicalImage - Export with proper headers
   DICOMMetadataExtractor - Patient/scan info
   SeriesSelector - Multi-series study navigation

2. Preprocessing Nodes:
   IntensityWindowing - HU window/level adjustment
   Reorientation - Standardize to RAS/LPS
   Resampling - Isotropic spacing
   CropROI - Focus on liver region
   NormalizeIntensity - Standardization

3. Visualization Nodes:
   MultiplanarViewer - Axial/Coronal/Sagittal slices
   3DRenderer - Volume rendering
   OverlayMask - Segmentation overlay with alpha blending
   BoundingBoxVisualizer - Detection boxes with labels
   SliceNavigator - Interactive slice browsing

4. AI Model Nodes:
   MONAIBundleLoader - Load pre-trained bundles
   LesionDetector - RetinaNet inference
   LesionSegmenter - UNet/SegResNet inference
   EnsemblePredictor - Multi-model fusion
   UncertaintyEstimator - Monte Carlo dropout

5. Interactive Refinement Nodes:
   DeepEditNode - Click-based refinement
   ManualAnnotation - Draw/erase tools
   LesionEditor - Add/remove detections
   ConfidenceThreshold - Filter by score

6. Analysis Nodes:
   VolumeCalculator - Lesion volume in mm³
   RadiomicsExtractor - Texture/shape features
   RECISTMeasurement - Longest diameter
   HUDensityAnalyzer - Mean/std HU values
   LesionCharacterizer - Classification features

7. Report Generation Nodes:
   StructuredReportBuilder - JSON/XML output
   VisualSummaryGenerator - Key images with annotations
   DICOMSRExporter - DICOM Structured Report
   PDFReportGenerator - Clinical report
   StatisticsTable - Quantitative summary

About me:
I have a medical degree and strong interest in medical imaging AI. Although not a practicing clinician, I understand clinical workflows and the needs of fellows, residents, and students in the medical field. I'm committed to creating a tutorial that bridges the gap between powerful AI tools and clinical usability, and if this proves useful I would like to contribute in a meaningful way.

Request for Feedback

I would greatly appreciate feedback from the MONAI community on:

1. Clinical relevance: Is this workflow relevant to current clinical needs?
2. Technical approach: Is ComfyUI integration (or node-based UI workflows in general) appropriate for MONAI tutorials?
3. Dataset choice: Please help me find publicly available liver lesion datasets.
4. Model selection: Recommendations for pre-trained models to use or study.

I look forward to your feedback and hope that this proposed tutorial meets with your approval.