How the ties are handled?

[cyuab]

For a given time series $T$, we can compute the Matrix Profile of $T$ by

mp = stumpy.stump(T, m=m)

And access the nearest neighbor and the corresponding distance of all the m-subsequences (i.e., length = $m$) by

mp.I_
mp.P_

So, if for a subsequence $T'$ at the position T_prime_idx, there are two nearest neighbors in $T$ that have the exact same distances from $T'$ (i.e., mp.P_[T_prime_idx]), which one will be picked for mp.I_[T_prime_idx]?
Any particular preferences? For example, the one closer to $T'$.
Sorry if it is written in the document.

[seanlaw]

Good question. There is no preference. Whichever nearest neighbor that was encountered first during the distance computation will be kept. Note that since this is done in parallel, it is not known what the order will be.

[cyuab]

Thanks for your reply. I agree it is challenging to know which one has been kept.
In some scenarios, I think it is useful to have a preference to handle the ties. For example, always keep the one that is closer to $T'$.

[seanlaw]

Out of curiosity, may I ask what it is that you are using STUMPY for?

[cyuab]

Thank you for your response and the hard work of all volunteers who have committed to STUMPY.

"we can't be everything for everyone"

Of course, I think the current implementation is powerful enough.

Recompute the distance profile for this subset of subsequences, compare the top-2 distances, and keep the desired subsequence

I think you mean matrix profile instead of distance profile to find out the top-2 smallest distances, right?
Using the trick mentioned in here, append a np.nan to the end of each subsequence, and apply matrix profile to this long sequence, and find the 2 smallest distances.
If you do mean distance profile, what is the "subsequence" that acts as a query?

Out of curiosity, may I ask what it is that you are using STUMPY for?

I am finding the motif of a given subsequence in gene expression data. I want to find the ones that are earlier than the given subsequence. If there are ties, I would like to retrieve the closer one.

[seanlaw]

I think you mean matrix profile instead of distance profile to find out the top-2 smallest distances, right?

No, I mean "distance profile". Since you were asking about ties and whether or not the one-nearest-neighbor is "closer" (by index location), you could verify this using the approach above by computing the distance profile for a subset of subsequences that were identified as having a one-nearest-neighbor below a desired distance threshold. Each of these identified subsequences would act as the query subsequence and you would compare it to all subsequences to its left, stumpy.mass(T[start_idx : stop_idx], T[ : start_idx - excl_zone]). Notice that we can save some computation by truncating the time series. Then, you can compare the values in the distance profile with the value computed from the left matrix profile. If there is more than one "smallest" distance then you may need to update your nearest neighbor location.

[cyuab]

Thank you so much again. This idea is neat!

[seanlaw]

I am finding the motif of a given subsequence in gene expression data. I want to find the ones that are earlier than the given subsequence. If there are ties, I would like to retrieve the closer one.

Interesting. Given that genes aren't continuous (floating point) values, does this mean that you are somehow able to encode your nucleotides as a numeric value? If so, how do you calculate a "distance" with these values? In case it matters, I use to be a biochemist (a long, long time ago but I now work in finance) and so I've always wondered how you'd effectively compute a "Euclidean distance" for genes (any reference to papers or best practices would be welcome).