Comparing Bag-of-Words and Embedding-Based Search in JavaScript

Welcome to our final lesson in this course about Text Representation Techniques for RAG systems! You’ve already explored the basics of Bag-of-Words (BOW) representations and experimented with sentence embeddings in earlier lessons. Now, we’re going to compare how these two methods differ in actual search scenarios. Think of this as a practical refresher on BOW and embeddings, but with an added focus on side-by-side comparison and deciding which approach might be best for different retrieval use cases.

Before diving into the code, let’s clarify why both methods — from straightforward word matching to deeper semantic modeling — are valuable.

• Lexical Overlap (BOW): This approach checks for exact word matches, making it easy to interpret how documents are scored. If your query has the phrase "external data," any document containing those exact words gets a higher score. It’s simple, transparent, and efficient for many tasks. But BOW can struggle with synonyms or varying phrasing.

• Semantic Similarity (Embeddings): Here, we focus on the overall meaning rather than specific words. Two differently phrased sentences can still be close in the embedding space if they convey the same idea. This approach excels at capturing nuances. However, it depends on a trained model and requires more computation.

In some real-world settings, you might even combine both: run a quick lexical match and then refine the results with a more precise semantic model. Let’s see how these methods look in code so you can start comparing results for yourself.

Below is an example of how to implement a BOW-based search workflow. We first build a vocabulary, then vectorize each document and the query according to how often each word appears.

function bowVectorize(text, vocab) {
    /**
     * Convert a text into a Bag-of-Words vector by counting how many times 
     * each token from our vocabulary appears in the text.
     */
    const vector = new Array(Object.keys(vocab).length).fill(0);
    text.toLowerCase().split(/\s+/).forEach(word => {
        const cleanWord = word.replace(/[.,!?]/g, '');
        if (vocab.hasOwnProperty(cleanWord)) {
            vector[vocab[cleanWord]] += 1;
        }
    });
    return vector;
}

function bowSearch(query, docs, vocab) {
    /**
     * Rank documents by lexical overlap using the BOW technique. 
     * The dot product between the query vector and each document vector 
     * indicates how many words they share.
     */
    const queryVec = bowVectorize(query, vocab);
    const scores = docs.map((doc, i) => {
        const docVec = bowVectorize(doc, vocab);
        const score = queryVec.reduce((sum, qVal, idx) => sum + qVal * docVec[idx], 0);
        return { index: i, score: score };
    });
    scores.sort((a, b) => b.score - a.score);
    return scores;
}

Let's break this down:

1. bowVectorize: Splits the text into words, applies some light cleanup (punctuation removal), and counts occurrences. If “external” appears once in the query, that contributes 1 to the corresponding position in the query vector.
2. bowSearch: Converts the query into a BOW vector, does the same for each document, and uses the dot product to measure shared token counts. Documents with many overlapping terms move to the top of the list.

This method is straightforward and fast for situations when exact word usage is critical. But what if your query is phrased differently than the document’s text? That’s where embeddings shine.