Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions tensor-groups/palm-detection-out.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Classification

- tensor-group: yes
- layer-type: output
- use-case: hand-detection

# Description

Palm Detection model outputs. This tensor group contains detection results for identifying palm/hand regions in images. The Palm Detection model is designed to locate hand bounding boxes and keypoints within an image.

Palm Detection Output Tensors:

| Name | shape |
|--- |--- |
| [detections] | N x 8 |

Where N is the number of detected palms/hands.

# Tensor Decoding Logic
Comment thread
dmorin1 marked this conversation as resolved.

```
Foreach detection in detections:
score = detection[0]
box_x = detection[1]
box_y = detection[2]
box_size = detection[3]
kp0_x = detection[4]
kp0_y = detection[5]
kp2_x = detection[6]
kp2_y = detection[7]

# Filter by confidence threshold (typically 0.60)
if score > confidence_threshold:
output_detection = [score, box_x, box_y, box_size, kp0_x, kp0_y, kp2_x, kp2_y]
```

## Calculate the orientation of the bounding box

```
d_x = kp2_x - kp0_x
d_y = kp2_y - kp0_y

angle = atan2 (-d_y, d_x )
```
**d_y** is negated because, in image coordinate systems, the y-axis typically increases downward, opposite to the conventional mathematical coordinate system.
**angle** is measured in radians.

# External References

* [MediaPipe Hand Detection](https://google.github.io/mediapipe/solutions/hands.html)
* [Palm Detection Model Architecture](https://github.com/google/mediapipe/blob/master/mediapipe/modules/hand_landmark/hand_detection.pbtxt)
* [Hand Gesture Recognition Using ONNX](https://github.com/PINTO0309/hand-gesture-recognition-using-onnx)

# Models

* [palm_detection_full_inf_post_192x192.onnx]

# Tensor Decoders

|Framework | Links |
|--- |--- |
|ONNX / Python | [Hand gesture recognition using ONNX](https://github.com/PINTO0309/hand-gesture-recognition-using-onnx) |


[detections]: /tensors/palm-detection-out-detections.md
[palm_detection_full_inf_post_192x192.onnx]: /models/palm_detection_full_inf_post_192x192.onnx
2 changes: 2 additions & 0 deletions tensor-id-register.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
|yolo-v8-segmentation-out-detections | [details](/tensors/yolo-v8-segmentation-out-detections.md) |
|yolo-v8-segmentation-out-protos | [details](/tensors/yolo-v8-segmentation-out-protos.md) |
|yolo-v8-out | [details](/tensors/yolo-v8-out.md) |
|palm-detection-out | [details](/tensors/palm-detection-out-detections.md) |

# Tensor Groups (Model Family)

Expand All @@ -21,3 +22,4 @@
|ssd-mobilenet-v1-variant-1-out | [details](/tensor-groups/ssd-mobilenet-v1-variant-1-out.md) |
|ultra-lightweight-face-detection-rfb-320-v1-variant-1-out | [details](/tensor-groups/ultra-lightweight-face-detection-rfb-320-v1-variant-1-out.md) |
|yolo-v8-segmentation-out | [details](/tensor-groups/yolo-v8-segmentation-out.md) |
|palm-detection-out | [details](/tensor-groups/palm-detection-out.md) |
71 changes: 71 additions & 0 deletions tensors/palm-detection-out-detections.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Classification

- tensor-group: no
- layer-type: output
- use-case: hand-detection
- part-of-tensor-groups:
- [palm-detection-out](/tensor-groups/palm-detection-out.md)

# Description

Palm detection predictions containing bounding box location, confidence score, and hand keypoints from the Palm Detection model. This tensor contains detection results for identifying palm/hand regions in images.

## Detections Tensor

- tensor-shape: N x 8
- tensor-datatype: float32

Where:
* N is the number of detected palms/hands

### Known Aliases
* pdscore_boxx_boxy_boxsize_kp0x_kp0y_kp2x_kp2y

### Encoding

Scheme: (Score, Box_X, Box_Y, Box_Size, Keypoint0_X, Keypoint0_Y, Keypoint1_X, Keypoint1_Y)

The tensor contains 8 channels with the following layout per detection:

|Channel 0 | Channel 1 | Channel 2 | Channel 3 | Channel 4 | Channel 5 | Channel 6 | Channel 7 |
|--- |--- |--- |--- |--- | --- |--- | --- |
| Score | Box_X | Box_Y | Box_Size | Keypoint0-x | Keypoint0-y | Keypoint1-x | Keypoint1-y |


Memory layout of tensor data (for detection i):

|Index | Symbol | Value | Comment |
|--- |--- |--- |--- |
| - | - | - | - |
|0 | pd_score | detection-i-confidence | tensor-start, detection-i-score |
|1 | box_x | detection-i-box-center-x | tensor-continue, detection-i-box-x |
|2 | box_y | detection-i-box-center-y | tensor-continue, detection-i-box-y |
|3 | box_size | detection-i-size | tensor-continue, detection-i-size |
|4 | keypoint0_x | detection-i-keypoint0-x | tensor-continue, detection-i-keypoint0-x |
|5 | keypoint0_y | detection-i-keypoint0-y | tensor-continue, detection-i-keypoint0-y |
|6 | keypoint1_x | detection-i-keypoint2-x | tensor-continue, detection-i-keypoint1-x |
|7 | keypoinp1_y | detection-i-keypoint2-y | tensor-end, detection-i-keypoint1-y |

### Details

- **pd_score** (Channel 0): Confidence score for the detection, in range [0, 1]
- **box_x, box_y** (Channels 1-2): Center coordinates of the detected palm bounding box in range [0,1] where 0 corresponds to the top (or left) and 1 to bottom (or right) edge of the image
- **box_size** (Channel 3): Side length of the bounding box, where the bounding box is square
- **Keypoint0** (Channels 4-5): Wrist keypoint
- **Keypoint1** (Channels 6-7): Middle finger joint to palm


# External References

* [MediaPipe Hand Detection](https://google.github.io/mediapipe/solutions/hands.html)
* [Palm Detection Model Architecture](https://github.com/google/mediapipe/blob/master/mediapipe/modules/hand_landmark/hand_detection.pbtxt)

# Models

* [palm_detection_full_inf_post_192x192.onnx](https://gitlab.collabora.com/gstreamer/onnx-models/-/blob/master/models/palm_detection_full_inf_post_192x192.onnx)

# Tensor Decoders

|Framework | Links |
|--- |--- |
|ONNX | [Hand gesture recognition using ONNX](https://github.com/PINTO0309/hand-gesture-recognition-using-onnx/blob/main/model/palm_detection/palm_detection.py) |