Vector Embeddings with OpenAI in Python

Introduction to Vector Embeddings

Welcome to our lesson on Vector Embeddings! In this lesson, you'll learn about one of the most powerful concepts in modern Natural Language Processing (NLP) - the ability to represent words and text as mathematical vectors. Vector embeddings are numerical representations that capture the meaning and relationships between words, allowing computers to understand and process human language in sophisticated ways. We'll explore how these embeddings work, why they're so important in modern AI applications, and learn how to generate them using OpenAI's powerful embedding models.

What Are Embeddings?

Embeddings are dense vector representations of data, often used in natural language processing (NLP) to represent words, phrases, or entire documents as lists of numbers. Each word or piece of text is converted into a vector of floating-point numbers, typically ranging from a few dozen to several hundred dimensions. Unlike simpler encoding methods like one-hot encoding (where each word is represented by a vector of mostly zeros with a single 1), embeddings capture semantic relationships in a lower-dimensional space, meaning that words with similar meanings end up being closer to each other in the vector space. The power of embeddings lies in their ability to capture multiple aspects of meaning simultaneously. Each dimension in the embedding vector can represent different semantic features - some dimensions might capture gender attributes, others might represent age-related concepts, and yet others might encode relationship hierarchies. This allows embeddings to represent complex relationships between words in a mathematical space. A classic example of this semantic representation is the relationship between words like "man," "woman," "king," and "queen." In the embedding space, these words are positioned such that the vector difference between "man" and "woman" (representing the concept of gender) is approximately equal to the vector difference between "king" and "queen." Additionally, interesting relationships like "queen" = "king" - "man" + "woman" show clearly how vector representations of these words carry semantic meaning. Below is the plot visualizing these relationships in a two-dimensional space.

Importance and Applications of Vector Embeddings

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Generating Vector Embeddings

To generate vector embeddings, we'll use the OpenAI library, which provides powerful models for creating embeddings. If you're working locally, ensure you have the necessary environment set up with access to the OpenAI API. You can store the API key in a .env file for security, load it using the load_dotenv() function, and pass it to the OpenAI() client. However, in the CodeSignal IDE, this setup is already configured for you, so to initialize the OpenAI client, just define OpenAI() as shown below:

Python

from openai import OpenAI

# Initialize OpenAI client
client = OpenAI()

def get_openai_embedding(text: str):
    response = client.embeddings.create(
        model="text-embedding-ada-002",
        input=text
    )
    # The embedding vector is in response.data[0].embedding
    return response.data[0].embedding

# Example usage
text = "Vector embeddings are powerful for semantic search."
embedding = get_openai_embedding(text)
print("Embedding dimension:", len(embedding))
print("First 10 values:", embedding[:10])

from openai import OpenAI

# Initialize OpenAI client
client = OpenAI()

def get_openai_embedding(text: str):
    response = client.embeddings.create(
        model="text-embedding-ada-002",
        input=text
    )
    # The embedding vector is in response.data[0].embedding
    return response.data[0].embedding

# Example usage
text = "Vector embeddings are powerful for semantic search."
embedding = get_openai_embedding(text)
print("Embedding dimension:", len(embedding))
print("First 10 values:", embedding[:10])

Output:

text

Embedding dimension: 1536
First 10 values: [-0.025095149874687195, -0.00015549981617368758, -0.002382089151069522, -0.022687843069434166, -0.0067814732901751995, 0.0034663851838558912, -0.002081176033243537, -0.03402503952383995, -0.00815659575164318, -0.041879042983055115]

Embedding dimension: 1536
First 10 values: [-0.025095149874687195, -0.00015549981617368758, -0.002382089151069522, -0.022687843069434166, -0.0067814732901751995, 0.0034663851838558912, -0.002081176033243537, -0.03402503952383995, -0.00815659575164318, -0.041879042983055115]

The core of embedding generation happens in the client.embeddings.create() method, where we specify the model and input text. We're using the text-embedding-ada-002 model, though other options are available such as text-embedding-3-large and text-embedding-3-small. Each model has its own characteristics and use cases.

When we call this function with a text string, it returns a vector of floating-point numbers representing the semantic meaning of that text. The length of this vector depends on the model used - for example, text-embedding-3-small produces 1536-dimensional vectors, while text-embedding-3-large produces 3072-dimensional vectors. You can also specify custom dimensions if needed for your specific use case.

Introduction to Vector Embeddings

What Are Embeddings?

Importance and Applications of Vector Embeddings

Join the 1M+ learners on CodeSignal

Generating Vector Embeddings

How pgvector Utilizes Embeddings

Conclusion and Next Steps