Implement information extraction solutions 5.1 Build retrieval and grounding pipelines
An Azure AI Search index stores 3,072-dimension float32 embeddings in an HNSW vector…
Study guide skill: Configure semantic search, hybrid search, and vector search for grounding
Question B3-028
An Azure AI Search index stores 3,072-dimension float32 embeddings in an HNSW vector field, and vector storage costs are too high. You want the largest reduction in vector index size while limiting the loss in relevance. What should you configure?
Select one answer.
Answer: C
Binary quantization stores each vector component as a single bit, a much bigger reduction than scalar quantization's int8 values, and it's especially effective for embeddings with more than 1,024 dimensions. You offset the lossy compression with oversampling and rescoring, which only HNSW supports. Exhaustive KNN doesn't support rescoring, and oversampling alone doesn't compress anything.
Deep dive
How it works
Azure AI Search has built-in quantization that shrinks vector fields, and Microsoft recommends it because it cuts both memory and disk storage for float16 and float32 embeddings. There are two kinds:
- Scalar quantization compresses each float into an int8 value. That makes the vector index about four times smaller.
- Binary quantization turns each component into a single bit (0 or 1). That makes the index up to 28 times smaller, and benchmark tests show up to 96% reduction in vector index size.
Binary works especially well for embeddings with more than 1024 dimensions and for embeddings centered around zero, which describes most popular OpenAI, Cohere and Mistral models. A 3,072-dimension field is a strong fit for binary quantization.
Quantization is lossy, so you offset it with rescoring. At query time, oversampling pulls in extra candidate vectors, and the results are rescored using the original full-precision vectors (or, for binary only, the binary candidates against the query vector). Rescoring options go in the index, and you can also turn it on at query time with the oversampling parameter. Setup steps: add vectorSearch.compressions, add a binaryQuantization configuration, create a vector profile that uses it, and assign that profile to a vector field.
The other options, and when you'd use them
- Switch to exhaustive KNN and add oversampling: Exhaustive KNN scans all vectors at query time, so it doesn't support rescoring, and oversampling is irrelevant there. Changing the algorithm also doesn't compress anything.
- Scalar (int8) quantization with no rescoring: Scalar is a smaller reduction (about fourfold) than binary, and skipping rescoring gives up the relevance protection the question asks for.
- Oversampling without any compression: Oversampling is part of rescoring compressed vectors. Without compression, there's nothing to shrink and nothing to rescore.
Exam tip
"Biggest storage cut on high-dimension embeddings" means binary quantization. "Protect relevance" means rescoring with oversampling, which needs HNSW, not exhaustive KNN.
References
Last reviewed: