Replace ItemCache sync.Map with Otter W-TinyLFU cache for better memory management

[zhangzhenghao]

Summary

This PR replaces the unbounded sync.Map in ItemCache with Otter's W-TinyLFU cache, providing automatic eviction based on access patterns.

Changes

1. ItemCache structure change:

   • sync.Map → otter.Cache[string, *data.Item]
   • Cache size is calculated as max(1024, itemCount/10)
   • itemCount is obtained from DataClient.CountItems(ctx) at cache creation time

2. API changes:

   • NewItemCache(client data.Database) → NewItemCache(client data.Database, itemCount int)
   • Uses Otter's GetIfPresent() and Set() methods

3. W-TinyLFU eviction policy:

   • Efficiently handles long-tail distribution data
   • Window region (1%) quickly evicts rarely accessed items
   • Main region retains popular items with frequency-based admission control

Benefits

Aspect	Before (sync.Map)	After (Otter W-TinyLFU)
Memory	Unbounded growth	Bounded, predictable
Eviction	None	Automatic W-TinyLFU
Hit rate	N/A	Optimized for long-tail
Concurrency	thread-safe	thread-safe

Cache Size Calculation

size := itemCount / 10  // 10% of total items
if size < 1024 {
    size = 1024  // minimum 1024 entries
}

Testing

All existing tests pass:

• worker/pipeline_test.go ✓
• worker/worker_test.go ✓

References

• Otter cache library - W-TinyLFU implementation
• Caffeine design - Original W-TinyLFU algorithm

[codecov]

Codecov Report

❌ Patch coverage is 81.25000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.88%. Comparing base (2241300) to head (4b42473).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
worker/pipeline.go	81.25%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1251      +/-   ##
==========================================
- Coverage   73.01%   72.88%   -0.14%     
==========================================
  Files          91       91              
  Lines       16694    16713      +19     
==========================================
- Hits        12189    12181       -8     
- Misses       3262     3287      +25     
- Partials     1243     1245       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.