[MOD-9685] Introduce SVS Basic Benchmarks #829
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
introduce BM_VecSimSVSTrain class with 2 methods: Train and TrainAsync add GoogleTest to benchmarks so we can use ASSERT_* API tieredIndexMock: possible to initialize with a specific thread count add train bemchmark to CI benchmark dispatcher
rename svs_training_fp32 ->svs_indices_training_fp32 add to bm_files.sh
move svs params init to CreateTieredSVSIndex only 5 iterations
add compressed index bm
remove some prints
move UNIT_AND_ITERATIONS and QUANT_BITS_ARGS to bm_vecsim_basics_Svs
… to bm_training_initialize.h define DATA_TYPE_INDEX_T in bm_svs_training_fp*.cpp remove th 10k and 50k for arm
fix include header for fp16
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #829 +/- ##
=======================================
Coverage 96.63% 96.63%
=======================================
Files 126 126
Lines 7379 7379
=======================================
Hits 7131 7131
Misses 248 248 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
we hard code the name of the data file based on quantBits
benchmark deletsion according to a when we exceed 0.5 index size this benchmark takes 20K ms (20s) for 500 vectors!!! that's a lot after revert - benchmark only gc to detrmine
runGC instead of delete label to not be depnd on consolidation_threshold that can't be controloed and runs for vrey very long!
Co-authored-by: BenGoldberger <[email protected]>
BenGoldberger
approved these changes
Nov 12, 2025
|
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 8.2
git worktree add -d .worktree/backport-829-to-8.2 origin/8.2
cd .worktree/backport-829-to-8.2
git switch --create backport-829-to-8.2
git cherry-pick -x 11cdc8b7a8435dfb2e62f37293f6dabd55b7465b |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a benchmarking suite for the SVS algorithm, including basic operations on loaded indices built on top of the existing training phase infrastructure.
New Benchmarks Added
BM_AddLabelOneByOne - Measures time to add individual vectors to a loaded SVS index one-by-one
BM_TriggerUpdateTiered - Measures time to move vectors from frontend (flat buffer) to backend (SVS index) in tiered index
BM_RunGC - Tests graph repairing after deletions