Recover from queue cache miss with oldest data instead of newest. #9711
Conversation
|
Should fix #9332 |
| exceptionOccured = exc; | ||
| consumerData.SafeDisposeCursor(logger); | ||
| // start from the entry at the low token, which is the first entry we know is still in the cache. | ||
| var tokenParsed = long.TryParse(exc.Low, out var lowTokenLong); |
There was a problem hiding this comment.
In the case we are working in it is using memory streams and this seems to work. However, for other tokens even the one I used below the EventSequenceTokenV2 and EventSequenceToken both have a ToString that wouldn't work for this.
A nicer option potentially is to have a way that queueCache.GetCacheCursor can start from the lowest without passing a StreamSequenceToken. I understand that queueCache.GetCacheCursor passing null starts with the newest message makes sense for case of a new subscription.
There was a problem hiding this comment.
This fix is definitely not correct and just appeared to be working as the string parsing doesn't match the sequence number ToString().
There was a problem hiding this comment.
The issue with the parsing has been fixed and tested to fix the problem, but of course the parsing is still not a nice solution.
Also, this has been testing with a failing case now.
1 participant
My goal is to fix issues with a stream that doesn't produce data very often. The problem is that the last sent value is no longer in the cache. However, the old value not being in the cache isn't really important. But when a new value comes for a stream that value is lost because when a QueueCacheMissException occurs and the cursor is set to start at the newest message and therefore skips the message that was just added, but is actually in the cache.
See https://github.com/BMagerMT/OrleansStreamingIssue as an example that produces this problem.
The issue is this fix is not great as it is parsing the token from the exception. However, there doesn't seem to be a way to create a cursor that starts with the oldest entries.
Microsoft Reviewers: Open in CodeFlow