Creating a Knowledge Base

Welcome back to Managing Data for GenAI with Bedrock Knowledge Bases! In this third lesson of our course, you're about to complete the architectural trinity that transforms raw documents into an intelligent, query-ready system. Having successfully built your vector storage infrastructure and populated it with document embeddings in our previous lessons, you now stand at the threshold of creating the sophisticated orchestration layer that makes your vector data truly accessible and useful for advanced AI applications.

Today marks a pivotal moment, as we'll create a Bedrock Knowledge Base — the intelligent bridge that connects your meticulously prepared vector embeddings to Amazon Bedrock's powerful retrieval and generation capabilities. Unlike the lower-level operations we've performed so far, this knowledge base represents a high-level abstraction that handles the complex choreography of query processing, similarity search, and context retrieval. By the end of this lesson, you'll have transformed your collection of Tech Company Inc.'s internal documents from static vector representations into a dynamic, production-ready knowledge system capable of powering sophisticated RAG applications and intelligent document search workflows.

Before writing our first line of code, let's develop a deep understanding of what a Bedrock Knowledge Base represents in the broader GenAI ecosystem. A knowledge base isn't simply a wrapper around your vector storage — it's a sophisticated orchestration service that manages the entire lifecycle of semantic search operations, from query embedding to result ranking and metadata handling. Think of it as the intelligent middleware that transforms mathematical vector operations into business-ready retrieval capabilities.

The architectural significance of a knowledge base becomes clear when you consider the alternative: manually coordinating embedding model invocations, implementing retrieval algorithms, managing similarity thresholds, handling result aggregation, and maintaining consistency between query and document embeddings. A Bedrock Knowledge Base abstracts these complexities behind a unified API that ensures your query embeddings use the exact same model and dimensions as your stored documents, implements advanced retrieval strategies that go beyond simple cosine similarity, and provides built-in integration points for other AWS services. This abstraction layer enables you to focus on building intelligent applications rather than wrestling with the mathematical intricacies of vector search, while still maintaining the flexibility to customize retrieval behavior through configuration parameters that we'll explore throughout this lesson.

From a cost perspective, Bedrock Knowledge Bases operate on a pay-per-use model that charges separately for vector storage (based on the amount of embedded data), retrieval queries, and embedding model invocations during both indexing and querying phases. This transparent pricing structure allows you to optimize costs by right-sizing your knowledge base for your specific query volume and data requirements, making it economical for both experimental prototypes and production-scale applications.

Creating a Bedrock Knowledge Base requires careful attention to AWS Identity and Access Management (IAM) configuration, specifically through the use of service roles. A service role is a special type of IAM role that allows AWS services to perform actions on your behalf, acting as the security bridge between Bedrock and your vector storage resources. Unlike user-based permissions that authenticate human operators, service roles enable automated, service-to-service interactions while maintaining strict security boundaries and audit trails. In our CodeSignal environment, the role creation and setup is already taken care of under the hood so that you can focus on Bedrock Knowledge Bases.

For our knowledge base to function properly, its service role needs a trust relationship policy that explicitly allows bedrock.amazonaws.com to assume the role, combined with permission policies that grant access to your S3 Vectors resources, the ability to invoke embedding models, and knowledge base management operations. The trust relationship ensures that only Amazon Bedrock can use this role, while the permission policies follow the principle of least privilege by granting only the specific actions required for knowledge base operations. This includes permissions like s3vectors:ListVectorBuckets and s3vectors:GetVectors for accessing your vector data, bedrock:InvokeModel for generating query embeddings, and various bedrock:*KnowledgeBase actions for management operations.

Our implementation begins by establishing the necessary AWS service connections and dynamically retrieving account information that we'll need for constructing resource identifiers:

import os
import uuid
import boto3

# Create Bedrock Agent client
bedrock_agent_client = boto3.client("bedrock-agent")

# Get AWS account ID and region using STS and environment
sts_client = boto3.client("sts")
account_id = sts_client.get_caller_identity()["Account"]

# AWS region name
REGION_NAME = os.getenv("AWS_REGION", "us-east-1")

The bedrock-agent client serves as our primary interface for all knowledge base operations, from initial creation through ongoing management and eventual deletion if needed. The Security Token Service (STS) provides a programmatic method to discover your AWS account ID at runtime through get_caller_identity(), eliminating the security risk and maintenance burden of hard-coding account-specific values. This dynamic retrieval pattern is particularly valuable in environments where code might be deployed across multiple AWS accounts or regions. The REGION_NAME configuration demonstrates defensive programming by checking for an environment variable first, then falling back to a sensible default — this approach ensures your code works seamlessly in CodeSignal's environment while remaining portable to other deployment contexts where the region might differ.

With our AWS context established, we now define the critical parameters that link our knowledge base to the existing vector infrastructure and specify the embedding model configuration:

# Name of the S3 Vectors index
VECTOR_INDEX_NAME = "bedrock-vector-index"

# Build the S3 Vectors index ARN as a constant
vector_index_arn = f"arn:aws:s3vectors:{REGION_NAME}:{account_id}:bucket/bedrock-vector-bucket/index/{VECTOR_INDEX_NAME}"

# Create a unique knowledge base name
KB_NAME = "bedrock-knowledge-base"

# IAM role ARN for knowledge base permissions
KB_ROLE_ARN = f"arn:aws:iam::{account_id}:role/kb-service-role"

# Set the embedding model ID to use
EMBEDDING_MODEL_ID = "amazon.titan-embed-text-v2:0"

# Set the number of dimensions for the embeddings
EMBEDDING_DIMENSIONS = 1024

The ARN (Amazon Resource Name) construction for vector_index_arn follows AWS's hierarchical naming convention, creating a globally unique identifier that precisely locates your S3 Vectors index within the vast AWS ecosystem. Notice the specific format: arn:aws:s3vectors: followed by region, account ID, and the path to your index — this structure enables AWS services to unambiguously reference resources across accounts and regions. The EMBEDDING_MODEL_ID specification of amazon.titan-embed-text-v2:0 ensures query consistency with the embeddings we generated in the previous lesson, while the :0 suffix indicates we're using the base version of this model. The dimension specification of 1024 must match exactly what we configured in our vector index; any mismatch would cause retrieval failures, since the mathematical operations underlying similarity search require vectors of identical dimensionality.

Now we execute the pivotal operation that transforms our separate components into a unified knowledge base system. The creation process requires careful structuring of nested configuration objects that define both the computational and storage aspects of our retrieval system:

try:
    # Create Bedrock Knowledge Base
    kb_response = bedrock_agent_client.create_knowledge_base(
        name=KB_NAME,
        description="Knowledge base using S3 Vectors for document retrieval",
        roleArn=KB_ROLE_ARN,
        knowledgeBaseConfiguration={
            "type": "VECTOR",
            "vectorKnowledgeBaseConfiguration": {
                "embeddingModelArn": (
                    f"arn:aws:bedrock:{REGION_NAME}::foundation-model/{EMBEDDING_MODEL_ID}"
                ),
                "embeddingModelConfiguration": {
                    "bedrockEmbeddingModelConfiguration": {
                        "dimensions": EMBEDDING_DIMENSIONS
                    }
                },
            },
        },
        storageConfiguration={
            "type": "S3_VECTORS",
            "s3VectorsConfiguration": {
                "indexArn": vector_index_arn,
            },
        },
        clientToken=str(uuid.uuid4()),  # Unique client token for idempotency
    )

The create_knowledge_base operation requires a sophisticated nested configuration structure that precisely defines both the computational and storage aspects of your retrieval system. Let's examine each configuration component and its role in establishing your knowledge base:

• Basic Parameters: The function begins with straightforward identification parameters including name for your knowledge base identifier, description for documentation purposes, and roleArn specifying the IAM service role that grants Bedrock the necessary permissions to access your vector storage and invoke embedding models.
• knowledgeBaseConfiguration: This top-level object specifies "type": "VECTOR" to indicate we're creating a vector-based knowledge base (as opposed to other potential types), then nests a vectorKnowledgeBaseConfiguration object containing two critical sub-components: the embeddingModelArn constructed using the specific AWS format with double colons :: before foundation-model to indicate this is an AWS-managed model rather than a custom deployment, and the embeddingModelConfiguration hierarchy that ensures our knowledge base knows exactly how to process queries by specifying 1024 dimensions within the bedrockEmbeddingModelConfiguration object.
• storageConfiguration: This section establishes the connection to your S3 Vectors infrastructure by specifying "type": "S3_VECTORS" and nesting an s3VectorsConfiguration object that contains the indexArn pointing to the vector index created in lesson one, creating the essential link between your knowledge base and the stored document embeddings.
• clientToken: This parameter provides idempotency protection through a unique UUID — if network issues cause this request to be retried, AWS will recognize the duplicate token and return the existing knowledge base rather than creating a duplicate, preventing resource proliferation and potential naming conflicts in your account.

After initiating the creation process, we need to capture the generated identifiers and verify the operational status of our new knowledge base:

    # Extract the Knowledge Base ID from the response
    knowledge_base_id = kb_response["knowledgeBase"]["knowledgeBaseId"]
    print(f"Knowledge Base ID: {knowledge_base_id}")

    # Check the status of the newly created Knowledge Base
    kb_status = bedrock_agent_client.get_knowledge_base(
        knowledgeBaseId=knowledge_base_id
    )
    
    # Get the current status of the Knowledge Base
    status = kb_status["knowledgeBase"]["status"]
    print(f"Knowledge Base Status: {status}")

except Exception as e:
    print(f"Error: {e}")

The knowledge base ID extracted from the creation response becomes your primary handle for all future operations — you'll use this identifier when querying documents, updating configurations, or integrating with other AWS services. The immediate status check through get_knowledge_base() provides visibility into the creation process, which is particularly important since knowledge base provisioning isn't instantaneous. When this code executes successfully, you'll see output similar to:

Knowledge Base ID: EX9OWR97HE
Knowledge Base Status: CREATING

The CREATING status indicates that AWS is actively provisioning your knowledge base infrastructure, a process that typically takes 30–60 seconds as it establishes connections between your vector storage, embedding model, and the retrieval APIs. Knowledge bases transition through several possible states during their lifecycle: CREATING during initial provisioning, ACTIVE when fully operational and ready for queries, UPDATING when configuration changes are being applied, DELETING during removal, and FAILED if provisioning encounters errors. Since our code doesn't implement a polling loop to wait for completion, we observe the initial CREATING state — in production scenarios, you might implement a wait loop that periodically checks the status until it transitions to ACTIVE, ensuring your knowledge base is fully ready before attempting retrieval operations.

Congratulations on successfully creating your first Bedrock Knowledge Base! You've mastered the essential skills of configuring service roles for secure AWS service interactions, constructing proper resource ARNs for cross-service integration, and orchestrating the complex creation process that transforms vector storage into an intelligent retrieval system. Your knowledge base now serves as the sophisticated interface layer that enables semantic search, context-aware responses, and seamless integration with the broader AWS AI ecosystem.

The upcoming practice section will challenge you to apply these concepts through hands-on exercises that reinforce your understanding of knowledge base configuration, error handling, and status monitoring. You'll have opportunities to experiment with different parameters, troubleshoot common issues, and build confidence in creating and managing these powerful GenAI infrastructure components that form the backbone of modern RAG applications.