Disallow dot_product and max_inner_product for int8_hnsw GPU type

[mayya-sharipova]

For int8_hnsw, during merges we get quantized vectors from Lucene files but dropping for each quantized vector its correction factor. For cosine and euclidean metrics this correction factor is not important, but for dot_product and max_inner_product metrics, they are important. It means that that currently for dot_product and max_inner_product metrics, GPU graph building doesn't work well, and may produce bad recall. This PR does the following:

• disallows max_inner_product for int8
• substitutes internally dot_product with cosine

Alternatives: for most datasets (really majority), we can substitute dot_product with cosine. But there are some datasets that require max_inner_product, and for this our "int8_hsnw" will not work, and "hnsw" should be used instead

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[elasticsearchmachine]

Hi @mayya-sharipova, I've created a changelog YAML for you.

[mayya-sharipova]

KnnIndexTester
opeanai: 2.6M docs; 1536 dims; 8 indexing threads

Notice low recall for dot_product on gpu, while recall recovers for cosine.

int8 dot_product

index_type	index_time( ms)	force_merge_time (ms)	QPS multiple segs	recall multiple segs	QPS force_merge 5 segs	recall 5 segs
cpu	697877	152161	95	0.84	148	0.84
gpu	235097	29671	117	0.71	185	0.48

int8 cosine

index_type	index_time( ms)	force_merge_time (ms)	QPS multiple segs	recall multiple segs	QPS force_merge 5 segs	recall 5 segs
cpu	739290	151696	96	0.85	151	0.83
gpu	286267	45025	99	0.99	136	0.99

---

But for other datasets, there is no difference:

hotpotqa-arctic: 5.2M docs; 768 dims; 8 indexing threads
int8 dot_product

index_type	index_time( ms)	force_merge_time (ms)	QPS multiple segs	recall multiple segs	QPS force_merge 5 segs	recall 5 segs
cpu	565207	177059	147	0.67	224	0.69
gpu	357982	58152	124	0.88	167	0.89

index_type	index_time( ms)	force_merge_time (ms)	QPS multiple segs	recall multiple segs	QPS force_merge 5 segs	recall 5 segs
cpu	600691	175769	154	0.68	217	0.68
gpu	303617	54235	117	0.87	160	0.86

[benwtrent]

I think this should just silently change to cosine when using the GPU integration to build the index, then using DOT_PRODUCT on search would work just fine.

this way we only disallow max-inner-product.

[mayya-sharipova]

Good suggestion, I will follow up on this.

[mayya-sharipova]

Addressed in 60e1dc9

[benwtrent]

I think this is good. I have one concern about element_type: byte vs. quantized element_type: float which both would use CuVSMatrix.DataType.BYTE right?

[elasticsearchmachine]

💔 Backport failed

Status	Branch	Result
❌	9.2	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 136881