Text-guided semantic segmentation of valid objects throughout the video (YOLOE + SAM2).
在整个视频中对有效物体进行文本引导的语义分割（YOLOE + SAM2）。
Type 算子类型: mapper
Tags 标签: gpu, hf, video
| name 参数名 | type 类型 | default 默认值 | desc 说明 |
|---|---|---|---|
sam2_hf_model |
<class 'str'> | 'facebook/sam2.1-hiera-tiny' |
|
yoloe_path |
<class 'str'> | 'yoloe-11l-seg.pt' |
The path to the YOLOE model. |
yoloe_conf |
<class 'float'> | 0.5 |
Confidence threshold for YOLOE object detection. |
torch_dtype |
<class 'str'> | 'bf16' |
The floating point type used for model inference. Can be one of ['fp32', 'fp16', 'bf16']. |
if_binarize |
<class 'bool'> | True |
Whether the final mask requires binarization. If 'if_save_visualization' is set to True, 'if_binarize' will automatically be adjusted to True. |
if_save_visualization |
<class 'bool'> | False |
Whether to save visualization results. |
save_visualization_dir |
<class 'str'> | DATA_JUICER_ASSETS_CACHE |
The path for saving visualization results. |
args |
'' |
||
kwargs |
'' |
VideoObjectSegmentingMapper(sam2_hf_model='facebook/sam2.1-hiera-tiny', yoloe_path='yoloe-11l-seg.pt', yoloe_conf=0.2, torch_dtype='bf16', if_binarize=True, if_save_visualization=False)Sample 1: 1 video
video4.mp4:
| main_character_list | |
|---|---|
| ['glasses', 'a woman', 'a window'] |
Sample 2: 1 video
video3.mp4:
| main_character_list | |
|---|---|
| ['a laptop'] |
Sample 1: empty
| segment_data | |
|---|---|
| [673, 3, 1, 360, 480] | |
| cls_id_dict | 3 |
| object_cls_list | |
| [3] | |
| yoloe_conf_list | |
| [3] | |
Sample 2: empty
| segment_data | |
|---|---|
| [1190, 1, 1, 640, 362] | |
| cls_id_dict | 1 |
| object_cls_list | |
| [1] | |
| yoloe_conf_list | |
| [1] | |