Issue with NMM Merging Mechanism for Dense Rice Grain Images

[SaiJeevanPuchakayala]

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Question

Issue with the stitching mechanism of detected images using Non-Maximum Merging (NMM) in the supervision library. The merging mechanism does not seem to be working correctly on dense rice grain images. The code slices a high-resolution image into smaller 512x512 images for detection and attempts to merge them back to the original 2048x2048 image. However, the merging results are not accurate for densely packed objects.

Steps to Reproduce:

import torch
from sahi.predict import get_sliced_prediction
from sahi import AutoDetectionModel
import cv2
import matplotlib.pyplot as plt
import numpy as np

# Callback function for inference
def callback(image: np.ndarray) -> sv.Detections:
    with torch.no_grad():  # Disable gradient calculation
        result = model(image, device='cuda')[0]  # Specify GPU if available
    return sv.Detections.from_ultralytics(result)

# Function to process an image and display the result
def process_image(image_path: str):
    try:
        image = cv2.imread(image_path)
        image = cv2.resize(image, (2048, 2048))
        if image is None:
            raise FileNotFoundError(f"Image at path {image_path} could not be loaded.")

        # Perform inference
        slicer = sv.InferenceSlicer(
            slice_wh=(512, 512),
            overlap_ratio_wh=(0.1, 0.1),
            thread_workers=10,
            iou_threshold=0.1,
            overlap_filter_strategy=sv.OverlapFilter.NON_MAX_MERGE,
            callback=callback)

        detections = slicer(image)

        # Filter detections based on confidence
        detections = detections[detections.confidence > 0.3]

        mask_annotator = sv.MaskAnnotator()
        annotated_image = mask_annotator.annotate(scene=image.copy(), detections=detections)

        # Draw contours around the masks and fit ellipses
        for i in range(detections.mask.shape[0]):
            mask = detections.mask[i].astype(np.uint8)
            contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
            cv2.drawContours(annotated_image, contours, -1, (0, 255, 0), 2)

        plt.figure(figsize=(20, 20))
        plt.imshow(cv2.cvtColor(annotated_image, cv2.COLOR_BGR2RGB))
        plt.axis('off')
        plt.show()

        # Print the number of detections
        print(f"Number of detections: {len(detections.confidence)}")

    except Exception as e:
        print(f"An error occurred: {str(e)}")
        raise

# Example usage
image_path = '/content/drive/MyDrive/Mixed Grains/test image .jpg'
process_image(image_path)

And here’s what I’m seeing: instead of nicely merged detections, I get overlapping and misaligned masks. The NMM just doesn't seem to handle these dense objects well.

Sample Image: The image used is a high-resolution dense rice grain image.

Expected Behavior: The sliced 512x512 images should be correctly merged back to the original 2048x2048 image with accurate detections.

Observed Behavior: The merging mechanism of NMM does not seem to work as expected on dense rice grain images. The detections are not accurately merged, leading to overlapping and misaligned masks.

What I’ve Tried:

• Image Size: Adjusted the image size and slice size.
• Overlap Ratio: Played around with different overlap ratios.
• Confidence Threshold: Tweaked the confidence threshold for detections.

Additional

No response