Skip to content

colblk: memory optimize DataBlockDecoder #5157

@jbowens

Description

@jbowens

The size of the DataBlockDecoder and cockroachkvs KeySeeker structs together determine the size of the block.Metadata we allocate alongside blocks in the block cache. Since this is allocated for every block in the block cache, its memory usage is significant. Reducing the size of the block.Metadata allows more blocks to be cached, improving cache hit rates.

Currently the DataBlockDecoder embeds the BlockDecoder struct: https://github.com/cockroachdb/pebble/blob/master/sstable/colblk/data_block.go#L859

This shouldn't be necessary. We should be able to use a BlockDecoder while initially constructing the block metadata, but then afterwards we don't need to retain it (having already constructed the relevant pointers in the block). It seems like at the very least, we could drop the []byte data slice from the resulting block.Metadata, conserving 24 bytes per block.

Jira issue: PEBBLE-1126

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions