Generating and Comparing Sentence Embeddings with Java

Introduction

Welcome back! This is the second lesson in our “Text Representation Techniques for RAG Systems” series. In our previous lesson, we introduced the Bag-of-Words (BOW) approach to converting text into numerical representations. Although BOW is intuitive and lays a solid foundation, it does not capture word order or deeper context. Picture a helpdesk system that retrieves support tickets. Without a solid way to represent text contextually, customers searching for “account locked” might miss relevant entries labeled “login blocked” because the system can't recognize these phrases as related. This gap in understanding could lead to frustrated users and unresolved queries. Today, we'll take a big step forward by learning to generate more expressive text embeddings — vectors that represent the semantic meaning of entire sentences. By the end of this lesson, you will know how to produce these embeddings and compare them with each other using cosine similarity .

Understanding Sentence Embeddings

Imagine you have sentences scattered across a high-dimensional space, where each sentence is a point, and closeness in this space reflects semantic similarity. Unlike BOW — which only counts word occurrences — sentence embeddings capture the relationship between words, making similar sentences land near each other in this space. This powerful feature is vital for Retrieval-Augmented Generation (RAG) systems, where retrieving text that is closest in meaning to a query drives more accurate responses. Sentence embeddings provide a more nuanced understanding of linguistic context, going beyond simple counts of word frequencies. For example, while a BOW model might treat the sentences " I enjoy apples " and " He likes oranges " as quite different, embeddings can capture that both sentences express a personal preference for fruit. This richer representation is especially helpful in complex applications such as semantic search, recommendation engines, and advanced conversational systems, where subtle differences in meaning can greatly impact the results.

Understanding The Cosine Similarity Function

\text{cosine similarity}(A, B) = \frac{A \cdot B}{\|A\|\|B\|}

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Comparing Sentence Embeddings with Cosine Similarity

Finally, let's compare sentences according to how similar they are. We can compute similarity scores for each pair of sentences using the cosineSimilarity function introduced before:

Java

public class CompareEmbeddings {

    public static void main(String[] args) {
        // Assuming embeddings are obtained as shown in the previous section
        for (int i = 0; i < embeddings.length; i++) {
            for (int j = i + 1; j < embeddings.length; j++) {
                double simScore = CosineSimilarity.cosineSimilarity(embeddings[i], embeddings[j]);
                System.out.printf("Similarity('%s' , '%s') = %.4f%n", i, j, simScore);
            }
        }
    }
}

// Output:
// Similarity('A Large Language Model is a Generative AI model for text generation.' , 'Bananas are yellow fruits.') = 0.5765
// Similarity('A Large Language Model is a Generative AI model for text generation.' , 'What's monkey's favorite food?') = 0.7257
// Similarity('Bananas are yellow fruits.' , 'What's monkey's favorite food?') = 0.9017

public class CompareEmbeddings {

    public static void main(String[] args) {
        // Assuming embeddings are obtained as shown in the previous section
        for (int i = 0; i < embeddings.length; i++) {
            for (int j = i + 1; j < embeddings.length; j++) {
                double simScore = CosineSimilarity.cosineSimilarity(embeddings[i], embeddings[j]);
                System.out.printf("Similarity('%s' , '%s') = %.4f%n", i, j, simScore);
            }
        }
    }
}

// Output:
// Similarity('A Large Language Model is a Generative AI model for text generation.' , 'Bananas are yellow fruits.') = 0.5765
// Similarity('A Large Language Model is a Generative AI model for text generation.' , 'What's monkey's favorite food?') = 0.7257
// Similarity('Bananas are yellow fruits.' , 'What's monkey's favorite food?') = 0.9017

In these results, pairs referencing similar topics (e.g., RAG and LLMs, or monkeys and bananas) often produce higher similarity scores, while pairs referencing different topics yield lower scores. Notice how “Bananas are yellow fruits.” and “What's monkey's favorite food?” have no overlapping words yet still show a relatively high similarity, highlighting how embeddings capture semantic meaning more effectively than a simpler BOW model, which primarily relies on word overlap.

Introduction

Understanding Sentence Embeddings

Understanding The Cosine Similarity Function

Join the 1M+ learners on CodeSignal

Loading a Pre-trained Model for Sentence Embeddings

Encoding Sentences into Embeddings

Comparing Sentence Embeddings with Cosine Similarity

Conclusion And Next Steps