Skip to content

Commit ef0e552

Browse files
authored
fix(level): change split key range right key to use ts=0 (#1932)
## Problem This old implementation is not incorrect, but it does not seem to follow the logic in its description. When a compaction is split into subcompaction jobs, each of them builds an iterator on all tables touched by the compaction and iterates over a certain `keyRange` defined in `addSplits`. In combination, they covers the key range of all the tables in the compaction. The right key of a key range is intended to be the right key of a bottom level table (`cd,bot`), It is intended to include all different versions of that key, as described in the comments. However, using `math.MaxUint64` will exclude those keys from this `keyRange` and include them in the next split. The reason is that the timestamp is encoded as `math.MaUint64-ts` in `y.KeyWithTs`, so a key with larger ts is actually smaller in `y.CompareKeys`. It can be corrected by using ts=0. Note that even using ts=math.MaxUint64 is not going to drop keys unlike what the comments above suggest. because those keys are covered in the subsequent split and the last split have an empty right key, iterating till the end of the tables being compacted. ## Solution Changed the timestamp used for split key range right key from `math.MaxUint64` to `0`. Updated the comments above it.
1 parent 599ccc6 commit ef0e552

File tree

1 file changed

+5
-6
lines changed

1 file changed

+5
-6
lines changed

levels.go

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1073,14 +1073,13 @@ func (s *levelsController) addSplits(cd *compactDef) {
10731073
return
10741074
}
10751075
if i%width == width-1 {
1076-
// Right should always have ts=maxUint64 otherwise we'll lose keys
1077-
// in subcompaction. Consider the following.
1076+
// Right is assigned ts=0. The encoding ts bytes take MaxUint64-ts,
1077+
// so, those with smaller TS will be considered larger for the same key.
1078+
// Consider the following.
10781079
// Top table is [A1...C3(deleted)]
10791080
// bot table is [B1....C2]
1080-
// This will generate splits like [A1 ... C2] . Notice that we
1081-
// dropped the C3 which is the last key of the top table.
1082-
// See TestCompaction/with_split test.
1083-
right := y.KeyWithTs(y.ParseKey(t.Biggest()), math.MaxUint64)
1081+
// It will generate a split [A1 ... C0], including any records of Key C.
1082+
right := y.KeyWithTs(y.ParseKey(t.Biggest()), 0)
10841083
addRange(right)
10851084
}
10861085
}

0 commit comments

Comments
 (0)