Skip to content

Commit 5f47864

Browse files
authored
fix: improve caching behavior of database integration (#444)
This improves the caching behavior of the Lazy Load data system. Previously, calling `AllFlags` or `AllSegments` updated an "allFlags" or "allSegments" key in the unscoped freshness tracker. That's correct, as those keys act as rate limiters for the "all" queries. It then upserted each individual item in the _unscoped_ freshness tracker. This is not optimal, as the unscoped tracker is not referenced when we read individual items. This results in cache misses for individual items, even after they were all refreshed. This change properly adds the individual items into the scoped tracker. Now, `AllFlags` or `AllSegments` should result in "priming" the cache for future individual item fetches. This does have the behavior of syncing the TTL dates of all items to a single point in time. ------------- Before: ``` [LaunchDarkly] lazy load via redis (JSON): get allFlags - cache miss [LaunchDarkly] lazy load via redis (JSON): get allSegments - cache miss [LaunchDarkly] lazy load via redis (JSON): get test - cache miss [LaunchDarkly] lazy load via redis (JSON): get my-boolean-flag - cache miss ``` After: ``` [LaunchDarkly] lazy load via redis (JSON): get allFlags - cache miss [LaunchDarkly] lazy load via redis (JSON): get allSegments - cache miss [LaunchDarkly] lazy load via redis (JSON): get test - cache hit [LaunchDarkly] lazy load via redis (JSON): get my-boolean-flag - cache hit ```
1 parent 4a7f1f9 commit 5f47864

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

libs/server-sdk/src/data_systems/lazy_load/lazy_load_system.hpp

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -145,8 +145,9 @@ class LazyLoad final : public data_interfaces::IDataSystem {
145145
std::function<
146146
data_interfaces::IDataReader::CollectionResult<Item>()> const&
147147
getter) const {
148-
// Storing an expiry time so that the 'all' key and the individual
149-
// item keys will expire at the same time.
148+
// The 'all' key and the individual item keys should expire
149+
// at exactly the same time, because there is no separate data item
150+
// for 'all' - it exists only to rate limit the refresh of all items.
150151
auto const updated_expiry = ExpiryTime();
151152

152153
// Refreshing 'all' for this item is always rate limited, even if
@@ -158,7 +159,7 @@ class LazyLoad final : public data_interfaces::IDataSystem {
158159

159160
for (auto item : *all_items) {
160161
cache_.Upsert(item.first, std::move(item.second));
161-
tracker_.Add(item.first, updated_expiry);
162+
tracker_.Add(item_kind, item.first, updated_expiry);
162163
}
163164
} else {
164165
status_manager_.SetState(

0 commit comments

Comments
 (0)