[MOD-10185] add SVS fp16 tests #740
Open
+3,398
−57
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This PR introduces a comprehensive test suite for FP16 data types in the SVS algorithm.
The new test file
test_svs_fp16.cpp
is adapted from the existingtest_svs.cpp
to specifically validate SVS functionality withfloat16
vectors across single-index, multi-index, and tiered index configuration.Key Changes
New Test File:
tests/unit/test_svs_fp16.cpp
Test Class Structure
The file introduces three specialized test classes:
FP16SVSTest
: Core SVS functionality tests for single-index scenariosFP16SVSMultiTest
: Multi-index SVS tests (inherits fromFP16SVSTest
)FP16SVSTieredIndexTest
: Tiered index SVS tests (inherits fromFP16SVSTest
)Type Definitions: runs
float16
with quantization modes:VecSimSvsQuant_NONE
VecSimSvsQuant_8
VecSimSvsQuant_8x8_LeanVec
Custom Vector Generation: Added
GenerateVector()
andGenerateAndAddVector()
methods specifically forfloat16
with proper FP32→FP16 conversionData Conversion: All vector operations use
vecsim_types::FP32_to_FP16()
andvecsim_types::FP16_to_FP32()
for proper type conversionTest Coverage
Total Test Count: 50 tests (31 core + 10 multi-index + 9 tiered index)
Core SVS Tests (
FP16SVSTest
- 31 tests)All essential SVS functionality tests adapted for FP16, including both adapted tests from the original suite and new FP16-specific tests:
Adapted from original test suite (26 tests):
svs_vector_add_test
,svs_vector_update_test
svs_bulk_vectors_add_delete_test
svs_indexing_same_vector
,svs_reindexing_same_vector
,svs_reindexing_same_vector_different_id
svs_batch_iterator
,svs_batch_iterator_non_unique_scores
,svs_batch_iterator_reset
,svs_batch_iterator_corner_cases
resizeIndex
,svs_empty_index
,test_delete_vector
svs_vector_search_test_ip
,svs_vector_search_test_l2
,svs_vector_search_test_cosine
preferAdHocOptimization
,batchIteratorSwapIndices
rangeQuery
,rangeQueryCosine
test_svs_info
,test_basic_svs_info_iterator
,test_dynamic_svs_info_iterator
testSizeEstimation
sanity_reinsert_1280
New FP16-specific tests (5 tests):
svs_vector_search_test
: Enhanced vector search test adapted for FP16test_get_distance
: Distance calculation validation for FP16quant_modes
: Comprehensive quantization mode validationtest_override_all
: Vector override functionality testingscalar_quantization_query
: Scalar quantization query validationMulti-Index Tests (
FP16SVSMultiTest
- 10 tests)Specialized tests for multi-index scenarios where multiple vectors can share the same label:
vector_add_multiple_test
: Adding multiple vectors with same labelvector_search_test
: Search functionality in multi-index contextsearch_more_than_there_is
: Edge case handling for search limitsfind_better_score
: Score optimization in multi-vector scenariosfind_better_score_after_pop
: Score optimization after vector removalreindexing_same_vector_different_id
: Reindexing with different IDstest_svs_info
: Multi-index debug information validationtest_basic_svs_info_iterator
: Multi-index debug iterator testingrangeQuery
: Range queries in multi-index contextsvs_batch_iterator_basic
: Batch iteration for multi-indexTiered Index Tests (
FP16SVSTieredIndexTest
- 9 tests)Tests for tiered index architecture with background indexing:
CreateIndexInstanceSingle
/CreateIndexInstanceMulti
: Index creation validationRangeTestSingle
/RangeTestMulti
: Range query testing for both configurationsKNNSearch
: K-nearest neighbor search in tiered contextdeleteVector
/deleteVectorMulti
: Vector deletion in tiered indexesBatchIteratorSingle
/BatchIteratorMulti
: Batch iteration for tiered indexesTests Excluded (17 tests)
Tests from the original
test_svs.cpp
that were intentionally not included, either because they are not relevant to For couldn't be reliably implemented with FP16 precision:testTimeoutReturn_topK
,testTimeoutReturn_range
,testTimeoutReturn_batch_iterator
test_svs_parameter_combinations_and_defaults
,test_svs_parameter_consistency_across_metrics
resolve_ws_search_runtime_params
,resolve_bc_search_runtime_params
,resolve_use_search_history_runtime_params
,resolve_epsilon_runtime_params
svs_test_inf_score
(infinity values not achievable with FP16)testInitialSizeEstimation
(covered bytestSizeEstimation
),joinSearchParams
(type not relevant),FitMemoryTest
(no-op),logging_runtime_params
Additional Infrastructure Changes
SVS Index Implementation Enhancements
Enhanced Vector Data Access (
src/VecSim/algorithms/svs/svs.h
)getStoredVectorDataByLabel()
: Added comprehensive implementation for retrieving stored vector data by label (only supported for non-compressed indices, both single and multi-index configurations)Query Result Debugging (
src/VecSim/query_result_definitions.h
)operator<<
forVecSimQueryResult
andVecSimQueryReply
to enable better test debugging#ifdef BUILD_TESTS
to keep production code cleanvalidateTopKSearchTest()
function now prints all results of the assertion fails:ASSERT_TRUE(allUniqueResults(res)) << *res;
)Test Infrastructure Improvements
Enhanced Test Utilities (
tests/unit/unit_test_utils.cpp
&.h
)validateSVSIndexAttributesInfo()
function to validate SVS index parameters againstsvsInfoStruct
valuesExtended Test Data Generation (
tests/utils/tests_utils.h
)populate_float16_vec()
function for generating random FP16 test vectors