Skip to content

Commit d815882

Browse files
committed
fix: improve exponential backoff calculation
Given the default `MinRetryBackoff` of 8 ms, the previous implementation for calculating the backoff duration was this: 1. Calculate exponential: ``` d := minBackoff << uint(retry) // Retry 0: d = 8ms // Retry 1: d = 16ms // Retry 2: d = 32ms // Retry 3: d = 64ms ``` 2. Replace with linear jitter: ``` d = minBackoff + time.Duration(rand.Int63n(int64(d))) // Retry 0: d = 8ms + random(0, 8ms) = 8-16ms // Retry 1: d = 8ms + random(0, 16ms) = 8-24ms // Retry 2: d = 8ms + random(0, 32ms) = 8-40ms // Retry 3: d = 8ms + random(0, 64ms) = 8-72ms ``` The average delays show this isn't really exponential: ``` Retry 0: avg = 8 + 4 = 12ms Retry 1: avg = 8 + 8 = 16ms (1.33x growth) Retry 2: avg = 8 + 16 = 24ms (1.5x growth) Retry 3: avg = 8 + 32 = 40ms (1.67x growth) ``` This is actually linear growth with exponential jitter range: ``` delay = constant_base + random(0, exponential_range) ``` As described in https://aws.amazon.com/ko/blogs/architecture/exponential-backoff-and-jitter/, we want: ``` d := minBackoff << uint(retry) d += random(0, d) // Retry 0: d = 8ms + random(0, 8ms) = 8-16ms // Retry 1: d = 16ms + random(0, 16ms) = 16-32ms // Retry 2: d = 32ms + random(0, 32ms) = 32-64ms // Retry 3: d = 64s + random(0, 64ms) = 64-128ms ``` Note that even with this change, Redis Cluster may still not have enough breathing room for cluster discovery. `MaxRetries`, `MinRetryBackoff`, and `MaxRetryBackoff` likely still need to be tweaked. Relates to #2046
1 parent 9c1655e commit d815882

File tree

1 file changed

+5
-4
lines changed

1 file changed

+5
-4
lines changed

internal/internal.go

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,14 @@ func RetryBackoff(retry int, minBackoff, maxBackoff time.Duration) time.Duration
1616

1717
d := minBackoff << uint(retry)
1818
if d < minBackoff {
19-
return maxBackoff
19+
d = minBackoff
2020
}
2121

22-
d = minBackoff + time.Duration(rand.Int63n(int64(d)))
22+
jitter := time.Duration(rand.Int63n(int64(d)))
2323

24-
if d > maxBackoff || d < minBackoff {
25-
d = maxBackoff
24+
d = d + jitter
25+
if d > maxBackoff {
26+
return maxBackoff
2627
}
2728

2829
return d

0 commit comments

Comments
 (0)