Skip to content

None check for descriptors and length check for knnMatches in gmc.py #124

@BrendanWooAdmetal

Description

@BrendanWooAdmetal

When running BoT-SORT, the following error appeared in gmc.py.

Traceback (most recent call last):
  File "/home/admetal/miniforge3/envs/toolkit2/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/admetal/miniforge3/envs/toolkit2/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/admetal/software/admetal_golf/scripts/gestures/gesture_network.py", line 357, in _track_thread_callback
    tracks = tracker.update(dets, frame)  # Run BoT-SORT tracking
  File "/home/admetal/software/admetal_golf/scripts/gestures/human_tracker.py", line 71, in update
    tracks = self.tracker.update(dets, frame)
  File "/home/admetal/software/admetal_golf/scripts/BoT_SORT/tracker/bot_sort_rknn.py", line 301, in update
    warp = self.gmc.apply(img, dets)
  File "/home/admetal/software/admetal_golf/scripts/BoT_SORT/tracker/gmc.py", line 68, in apply
    return self.applyFeaures(raw_frame, detections)
  File "/home/admetal/software/admetal_golf/scripts/BoT_SORT/tracker/gmc.py", line 175, in applyFeaures
    for m, n in knnMatches:
ValueError: not enough values to unpack (expected 2, got 1)

Examination of the code showed that the issue came from here:

knnMatches = self.matcher.knnMatch(self.prevDescriptors, descriptors, 2)

The issue is that knnMatches will only find up to 2 matches. It may be fewer, according to OpenCV's documentation.

"Each matches[i] is k or less matches for the same query descriptor."

This means that if only 1 match is found, the following line will fail as it tries to unpack 1 value into 2.

for m, n in knnMatches:

A similar issue can occur if the descriptors are None.

knnMatches = self.matcher.knnMatch(self.prevDescriptors, descriptors, 2)

This will cause the line above to throw an error.

I don't have a full understanding of the algorithm, so I'm not sure if the following fixes are acceptable, but I propose some changes:

  1. Check if descriptors is None before using it in gmc.py line 154.
    If not, set knnMatches to an empty list so it is caught by the empty matches case in gmc.py line 163.
    I'm not sure if this is the correct way to handle this case.
        # Check if there are features
        if descriptors is None:
            knnMatches = []
        else:
            # Match descriptors.
            knnMatches = self.matcher.knnMatch(self.prevDescriptors, descriptors, 2)
  1. Check the length of knnMatches in gmc.py line 171 and ignore it if there are less than 2 matches.
        for knnmatch in knnMatches:
            # Safety check, as knnMatches will contain k==2 or fewer matches
            if len(knnmatch) == 2:
                m, n = knnmatch
                if m.distance < 0.9 * n.distance:
                    prevKeyPointLocation = self.prevKeyPoints[m.queryIdx].pt
                    currKeyPointLocation = keypoints[m.trainIdx].pt

                    spatialDistance = (prevKeyPointLocation[0] - currKeyPointLocation[0],
                                    prevKeyPointLocation[1] - currKeyPointLocation[1])

                    if (np.abs(spatialDistance[0]) < maxSpatialDistance[0]) and \
                            (np.abs(spatialDistance[1]) < maxSpatialDistance[1]):
                        spatialDistances.append(spatialDistance)
                        matches.append(m)

Again, I don't fully understand the algorithm, and am unsure if I am handling these cases correctly.
They come up rarely; I only experienced them each once and was unable to replicate them under normal conditions, but I feel safety checks may help.

Can someone please check these and confirm if these are acceptable ways to discard or handle invalid data?

I can submit a pull request after if it is deemed necessary.
That being said, I am aware this repository is a few years old and may not be maintained.

In any event, I hope this may help others that experience the same issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions