Query Latency in Vector Search

Welcome to the first lesson of our course on Optimizing and Scaling Qdrant for Vector Search. In this lesson, we will explore the concept of query latency in vector search systems and its significance in providing a seamless user experience. Query latency refers to the time it takes for a search query to return results. In vector search systems, reducing this latency is crucial for ensuring efficient and responsive interactions. One effective method to achieve this is by precomputing nearest neighbors, which allows us to quickly retrieve relevant results without recalculating distances for every query. This lesson will guide you through the process of implementing precomputed nearest neighbors using a vector database.

A vector database is a system designed to store, index, and retrieve high-dimensional vectors efficiently. These vectors are typically generated from text, images, or other data using embedding functions. An embedding function transforms data (such as text) into a numerical vector representation, which can then be used for similarity searches. By leveraging a vector database and embedding functions, we can efficiently manage our vector data and perform nearest neighbor searches.

Before we can precompute neighbors, we first need to set up our Qdrant collection, encode our documents, and upload them as vectors.

import numpy as np
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity
from qdrant_client import QdrantClient
from qdrant_client.http import models

# Configuration
collection_name = "vector-search"

# Load the embedding model
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

# Initialize Qdrant client (in-memory for demo)
client = QdrantClient(":memory:")

# Create collection with some demo documents
vector_dim = model.get_sentence_embedding_dimension()
if client.collection_exists(collection_name):
    client.delete_collection(collection_name)

client.create_collection(
    collection_name=collection_name,
    vectors_config=models.VectorParams(size=vector_dim, distance=models.Distance.COSINE)
)

docs = [
    {"id": 0, "content": "AI in computing is evolving fast."},
    {"id": 1, "content": "Advances in machine learning improve healthcare."},
    {"id": 2, "content": "Traveling in Europe during summer is fun."},
    {"id": 3, "content": "Quantum computing and AI may intersect soon."},
    {"id": 4, "content": "Solar energy adoption is growing worldwide."},
]

points = [
    models.PointStruct(
        id=doc["id"],
        vector=model.encode(doc["content"]).tolist(),
        payload={"content": doc["content"]}
    )
    for doc in docs
]

client.upsert(collection_name=collection_name, points=points)

Explanation:

• We import the necessary libraries for embeddings, similarity calculation, and Qdrant interaction.
• We load a pre-trained sentence transformer model to convert text into vectors.
• We initialize a Qdrant client (using in-memory storage for demonstration).
• We create a collection in Qdrant with the appropriate vector size and cosine distance metric.
• We define a small set of demo documents, encode them into vectors, and upload them to the Qdrant collection.

After uploading the data, we fetch it back with vectors and payloads, and then calculate pairwise similarities.

# Fetch all documents (with vectors) via scroll
all_docs = []
points, next_page = client.scroll(
    collection_name=collection_name,
    with_vectors=True,
    with_payload=True,
    limit=100
)
while points:
    all_docs.extend(points)
    if next_page is None:
        break
    points, next_page = client.scroll(
        collection_name=collection_name,
        with_vectors=True,
        with_payload=True,
        offset=next_page,
        limit=100
    )

# Precompute nearest neighbors
def precompute_neighbors(points, top_k=3):
    ids = [str(p.id) for p in points]
    embeddings = np.array([p.vector for p in points])

    similarity_matrix = cosine_similarity(embeddings, embeddings)

    neighbors = {
        ids[i]: sorted(
            [(doc_id, float(min(score, 1.0))) for doc_id, score in zip(ids, similarity_matrix[i]) if doc_id != ids[i]],
            key=lambda x: x[1],
            reverse=True
        )[:top_k]
        for i in range(len(ids))
    }
    return neighbors

precomputed_neighbors = precompute_neighbors(all_docs)
print("Precomputed nearest neighbors stored.")

Explanation:

• We use Qdrant's scroll method to fetch all documents, including their vectors and payloads.
• We define a function precompute_neighbors that:
  • Extracts the IDs and vectors from the points.
  • Computes the cosine similarity matrix for all document pairs.
  • For each document, sorts the other documents by similarity and selects the top 3 nearest neighbors (excluding itself).
• We store the precomputed neighbors in a dictionary for fast lookup.
• The print statement confirms that the neighbors have been precomputed.

Example Output:

Precomputed nearest neighbors stored.

Finally, we can quickly fetch the top neighbors for any document without recalculating similarity on the fly.

# Example: Print the first document and its neighbors
first_id = next(iter(precomputed_neighbors))
print(f"Document ID: {first_id}")
print("Top 3 Nearest Neighbors:")
for neighbor_id, similarity in precomputed_neighbors[first_id]:
    print(f"Neighbor ID: {neighbor_id}, Similarity: {similarity:.4f}")

Explanation:

• We select the first document ID from our precomputed neighbors.
• We print the document ID and then iterate through its top 3 nearest neighbors, printing each neighbor's ID and similarity score.

Example Output:

Document ID: 0
Top 3 Nearest Neighbors:
Neighbor ID: 3, Similarity: 0.7821
Neighbor ID: 1, Similarity: 0.6543
Neighbor ID: 4, Similarity: 0.3125

(Note: The actual similarity values may vary depending on the embedding model and data.)

In this example:

• We store a small set of demo documents in Qdrant.
• We retrieve all documents and their vectors using scroll.
• We compute cosine similarity across all pairs and store the top-3 nearest neighbors for each document.
• When a query comes in, we can instantly look up the neighbors instead of recalculating everything from scratch.

In this lesson, we explored the concept of query latency and how precomputing nearest neighbors can help reduce it in vector search systems. We learned how vector databases and embedding functions are used to manage vector data and perform efficient similarity searches. By precomputing nearest neighbors, we can significantly improve the performance of our search system. As you move on to the practice exercises, you will have the opportunity to reinforce these concepts and apply them to real-world scenarios. Remember, reducing query latency is crucial for providing a seamless user experience, and precomputing nearest neighbors is a powerful technique to achieve this.