New feature: tombsweeper
The main new feature in the release is the 'tombsweeper'. When a record is deleted in a Lightning Stream installation, we do not actually remove the record, but instead write a deletion marker. Up until now these deletion markers were never deleted, leading to unbounded database growth in installations that see delete-heavy workloads. The new tombsweeper can clean these deletion markers after a configured interval.
WARNING: This is DISABLED by default, because enabling this functionality requires careful consideration. When enabled, you MUST make sure that no instance that has been offline for longer than the sweeper retention_days will ever reconnect. If this does happen, old entries that have been deleted may be resurrected causing anything from undesired results to database state corruption. Be especially careful about development or testing systems that only come online occasionally.
For small personal installations with infrequent changes, it is probably better NOT to enable this functionality, and accept that deleted entries accumulate over time. Only consider this you have frequent updates and deletion markers would actually become a problem over time. And in that case, be very conservative with the retention_days setting, the longer the better.
What's Changed
- Tombsweeper: clean stale delete markers by @wojas in #89
- Rename module to github.com/PowerDNS/lightningstream/lmdbenv by @wojas in #76
- GitHub Actions workflow for building and pushing multi-platform Docker images to Docker Hub by @joel-ling in #78
- Upgrade to Go 1.24 & CI fixes by @wojas in #87
- Add hooks by @wojas in #88
- Module upgrades, including lmdb-go to 1.9.3 by @wojas in #91
- Add Minio to github actions to use in test suite by @verscheures in #77
New Contributors
- @joel-ling made their first contribution in #78
- @verscheures made their first contribution in #77
Full Changelog: v0.4.3...v0.5.0
