Ultralytics YOLO with Additional Knowledge Distillation Capability

Core Idea

How to Run

Example:

pip install -r requirements.txt

from ultralytics import YOLO

teacher_model = YOLO("<teacher-path>")

student_model = YOLO("yolo11n.pt)

student_model.train(
    data="<data-path>",
    teacher=teacher_model.model, # None if you don't wanna use knowledge distillation
    distillation_loss="cwd",
    epochs=100,
    batch=16,
    workers=0,
    exist_ok=True,
)

Credits

• Channel-Wise Distillation Loss Implementation: https://github.com/pppppM/mmdetection-distiller
• Mask Generation Distillation Loss Implementation: https://github.com/yzd-v/MGD
• YOLOv5 Knowledge Distillation Implementation: https://github.com/wonbeomjang/yolov5-knowledge-distillation

@software{Jocher_Ultralytics_YOLO_2023,
  author = {Jocher, Glenn and Qiu, Jing and Chaurasia, Ayush},
  license = {AGPL-3.0},
  month = jan,
  title = {{Ultralytics YOLO}},
  url = {https://github.com/ultralytics/ultralytics},
  version = {8.0.0},
  year = {2023}
}