Scaling Up RAG with Vector Databases: Document Chunking in JavaScript

Introduction

Welcome to the first lesson of our course on " Scaling Up RAG with Vector Databases "! This is the third course in our series, building on the foundations laid in the previous courses. Course 1 introduced a basic RAG pipeline, providing a foundational understanding of how retrieval and generation can be combined. Course 2 focused on text representation, with a particular emphasis on embeddings. In this course, we'll focus on scaling your Retrieval-Augmented Generation (RAG) system by building and querying a vector database. You'll learn to preprocess documents, store chunk embeddings in ChromaDB, retrieve relevant chunks using advanced techniques like compound metadata filters and weighting, and construct prompts that can handle multiple context chunks. Additionally, we'll cover managing updates to your collection and large-scale ingestion using batch strategies. Our journey begins with document chunking, a crucial preprocessing step that enhances the efficiency and effectiveness of vector databases in RAG systems. By the end of this lesson, you'll be able to break down a lengthy document into discrete segments, each tagged with essential metadata, paving the way for robust retrieval and storage in vector databases.

Understanding Document Chunking

When dealing with Retrieval-Augmented Generation, you typically feed chunks of text — along with certain metadata — into downstream components, like embedding models or vector search engines. This document chunking process is crucial for several reasons: Manageability : Smaller segments are easier to process. Many models have a maximum context length, meaning they can only handle a certain number of tokens at a time. If the input text exceeds this limit, the model may become inefficient or unable to process the request, leading to errors or truncated outputs. Additionally, even if a model can technically handle larger contexts, performance may degrade, resulting in slower processing times and reduced accuracy. Context Preservation : A well-sized chunk still retains enough local context to be meaningful. Chunks should be neither too large (leading to potential memory issues) nor too small (risking the loss of context). Enhanced Retrieval : When text is split rationally, you can retrieve only the relevant segments instead of searching through entire documents — this shortens query times and boosts accuracy. Think of a real-world example: If you have a large reference manual, you wouldn't read the whole thing to answer a single question. Instead, you'd look for the exact section (or chunk) that pertains to your query. The same principle carries into text retrieval on a computer.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Adding Meaningful Metadata

So far, we have a method for chunking text, but in RAG systems, we often need to know where each piece came from and what it represents. That's where metadata comes in. Below is an example function that demonstrates how to loop through a structured dataset of documents, chunk each one, and store metadata for later use:

JavaScript

function loadAndChunkDataset(data, chunkSize = 10) {
    /**
     * Iterates over a structured dataset of documents, splits each into chunks,
     * and associates metadata (doc_id and chunk_id) with every piece.
     */
    const allChunks = [];
    data.forEach(doc => {
        const docId = doc.id;
        const docText = doc.content;

        // Create smaller text segments from the original document
        const docChunks = chunkText(docText, chunkSize);

        // Label each chunk with its source identifier
        docChunks.forEach((chunkStr, chunkId) => {
            allChunks.push({
                doc_id: docId,
                chunk_id: chunkId,
                text: chunkStr
            });
        });
    });
    return allChunks;
}

function loadAndChunkDataset(data, chunkSize = 10) {
    /**
     * Iterates over a structured dataset of documents, splits each into chunks,
     * and associates metadata (doc_id and chunk_id) with every piece.
     */
    const allChunks = [];
    data.forEach(doc => {
        const docId = doc.id;
        const docText = doc.content;

        // Create smaller text segments from the original document
        const docChunks = chunkText(docText, chunkSize);

        // Label each chunk with its source identifier
        docChunks.forEach((chunkStr, chunkId) => {
            allChunks.push({
                doc_id: docId,
                chunk_id: chunkId,
                text: chunkStr
            });
        });
    });
    return allChunks;
}

Here's how it works step-by-step:

We iterate over data, which is a collection of different text entries in the dataset.
For each entry, the function extracts the id and content fields to process.
We apply the earlier chunkText function to split the content into multiple pieces.
We then store relevant information — doc_id and chunk_id — along with the chunked text. This means every piece of text is traceable back to its origin, which can be crucial for retrieval and filtering in a vector database.

Note that in docChunks.forEach((chunkStr, chunkId) => {, chunkId automatically tracks the index of each chunk in the array, which is useful for ordering. However, if you later modify chunking logic or reorder entries for storage or scoring, this index may lose meaning. To maintain more precise traceability, consider also storing a start_offset—the original word or character position where the chunk begins. This helps reconstruct the original context accurately, especially in systems where chunk ordering might change.

Practical Usage Example

In this lesson, we'll use two sample documents to demonstrate the chunking process and to illustrate the concepts discussed. In the practice section, we'll be using a more realistic dataset for hands-on exercises; it can be found at src/data/corpus.json.

When we run the following code snippet with chunkSize = 10:

JavaScript

const chunkedDocs = loadAndChunkDataset(data, 10);
console.log(`Loaded and chunked ${chunkedDocs.length} chunks from dataset.`);
chunkedDocs.forEach(doc => {
    console.log(doc);
});

const chunkedDocs = loadAndChunkDataset(data, 10);
console.log(`Loaded and chunked ${chunkedDocs.length} chunks from dataset.`);
chunkedDocs.forEach(doc => {
    console.log(doc);
});

we get this output:

JSON

Loaded and chunked 7 chunks from dataset.
{ doc_id: 1, chunk_id: 0, text: 'Hello world! This is a sample document used for testing' }
{ doc_id: 1, chunk_id: 1, text: 'chunkText function.' }
{ doc_id: 2, chunk_id: 0, text: 'Another sample document. This is used for verifying the chunking' }
{ doc_id: 2, chunk_id: 1, text: 'of text in multiple documents. It includes additional sentences to' }
{ doc_id: 2, chunk_id: 2, text: 'provide a more comprehensive test case. By having a longer' }
{ doc_id: 2, chunk_id: 3, text: 'document, we can better assess how the chunking function performs' }
{ doc_id: 2, chunk_id: 4, text: 'when dealing with more extensive content.' }

Loaded and chunked 7 chunks from dataset.
{ doc_id: 1, chunk_id: 0, text: 'Hello world! This is a sample document used for testing' }
{ doc_id: 1, chunk_id: 1, text: 'chunkText function.' }
{ doc_id: 2, chunk_id: 0, text: 'Another sample document. This is used for verifying the chunking' }
{ doc_id: 2, chunk_id: 1, text: 'of text in multiple documents. It includes additional sentences to' }
{ doc_id: 2, chunk_id: 2, text: 'provide a more comprehensive test case. By having a longer' }
{ doc_id: 2, chunk_id: 3, text: 'document, we can better assess how the chunking function performs' }
{ doc_id: 2, chunk_id: 4, text: 'when dealing with more extensive content.' }

This output clearly illustrates both the benefits and limitations of our simple chunking approach. While we've successfully broken down the documents into smaller pieces with proper metadata tracking, the quality of some chunks is problematic. For example, chunk 1 of document 1 contains only the fragment 'chunkText function.', which lacks sufficient context to be meaningful on its own. Similarly, other chunks cut across sentence boundaries, creating segments that might be difficult for retrieval systems to properly interpret.

These issues highlight why more sophisticated chunking strategies that respect semantic boundaries (like sentences or paragraphs) are essential for context preservation in production RAG systems. Poor chunking can significantly reduce the effectiveness of retrieval, as chunks without adequate context may not match relevant queries or might provide incomplete information. In the practice section, we'll explore more advanced techniques that better preserve context and create more meaningful chunks.

Introduction

Understanding Document Chunking

Join the 1M+ learners on CodeSignal

Splitting Large Text into Chunks

Inspecting the Input Data

Adding Meaningful Metadata

Practical Usage Example

Conclusion and Next Steps