Real-Time Endpoint Deployment

Introduction & Overview

Welcome to another pivotal moment in your SageMaker deployment journey! Over the past three lessons, you've mastered serverless deployments, learning how to deploy various model types to cost-effective endpoints that scale to zero and charge only for actual usage. Now you're ready to explore a fundamentally different paradigm: real-time endpoints that maintain persistent, always-on infrastructure for consistent, low latency responses. By the end of this lesson, you'll understand how to configure persistent endpoints, select appropriate instance types, test deployments with production-like workflows, and monitor endpoint performance—completing your understanding of SageMaker's deployment options.

Understanding Real-Time Endpoints Architecture

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Finding and Preparing the Trained Model

The deployment process for real-time endpoints builds upon the concepts you've learned in previous lessons while introducing new configuration options specific to persistent infrastructure. As you've done before, we'll start by establishing a SageMaker session and locating a trained model, but this time you'll configure the deployment for always-on availability rather than serverless scaling.

Python

import sagemaker
from sagemaker.sklearn.estimator import SKLearn

# Create a SageMaker session
sagemaker_session = sagemaker.Session()

# List most recent completed training job
training_jobs = sagemaker_session.sagemaker_client.list_training_jobs(
    SortBy='CreationTime',                 # Sort jobs by creation time
    SortOrder='Descending',                # Newest jobs first
    StatusEquals='Completed',              # Only include completed jobs
    NameContains='sagemaker-scikit-learn'  # Filter to sklearn estimator jobs
)

# Extract the name of the latest training job of the list
TRAINING_JOB_NAME = training_jobs['TrainingJobSummaries'][0]['TrainingJobName']

import sagemaker
from sagemaker.sklearn.estimator import SKLearn

# Create a SageMaker session
sagemaker_session = sagemaker.Session()

# List most recent completed training job
training_jobs = sagemaker_session.sagemaker_client.list_training_jobs(
    SortBy='CreationTime',                 # Sort jobs by creation time
    SortOrder='Descending',                # Newest jobs first
    StatusEquals='Completed',              # Only include completed jobs
    NameContains='sagemaker-scikit-learn'  # Filter to sklearn estimator jobs
)

# Extract the name of the latest training job of the list
TRAINING_JOB_NAME = training_jobs['TrainingJobSummaries'][0]['TrainingJobName']

This initial setup should look familiar from your previous lessons, as you're using the same approach to locate your most recent completed training job. The NameContains='sagemaker-scikit-learn' filter helps you identify estimator-based training jobs, which are ideal for demonstrating real-time endpoint deployment because they can be easily attached using the SKLearn.attach() method you learned before.

With your training job identified, you're ready to move on to configuring the persistent infrastructure that will distinguish this deployment from the serverless endpoints you've worked with previously.

Configuring Real-Time Endpoint Resources

The real-time endpoint configuration introduces new parameters that define the persistent infrastructure characteristics. Unlike serverless endpoints that automatically manage scaling and resource allocation, real-time endpoints require you to explicitly specify the compute resources that will host your model.

Python

# Unique name to assign to the deployed real-time endpoint
ENDPOINT_NAME = "california-housing-realtime"
# Type of instance to use for the endpoint
INSTANCE_TYPE = "ml.m5.large"
# Number of instances to launch for the endpoint
INSTANCE_COUNT = 1

# Unique name to assign to the deployed real-time endpoint
ENDPOINT_NAME = "california-housing-realtime"
# Type of instance to use for the endpoint
INSTANCE_TYPE = "ml.m5.large"
# Number of instances to launch for the endpoint
INSTANCE_COUNT = 1

These configuration parameters control the fundamental characteristics of your real-time endpoint infrastructure:

• INSTANCE_TYPE: Specifies the EC2 instance type that will host your model. The ml.m5.large instance type provides a robust combination of compute, memory, and network performance that's well-suited for production-grade machine learning inference. This instance type offers 2 vCPUs and 8 GB of memory, which provides ample resources for scikit-learn models while ensuring consistent performance under load. The M5 family is optimized for general-purpose computing with balanced resources, making it an excellent choice for a wide range of machine learning workloads.

• INSTANCE_COUNT: Determines how many instances will be deployed to serve your model. Since real-time endpoints charge for all running instances continuously (unlike serverless endpoints that only charge for actual usage), each additional instance directly multiplies your hourly costs. Starting with a single instance minimizes costs during development and testing phases while still providing the persistent availability that real-time endpoints offer. Production deployments often use multiple instances to provide redundancy and handle higher request volumes, but this comes with proportionally higher costs as SageMaker automatically distributes incoming requests across all available instances.

Understanding these parameters is crucial for optimizing both performance and cost in your real-time endpoint deployments. The instance type selection directly impacts your model's response time and throughput capabilities, while the instance count affects your endpoint's ability to handle concurrent requests and maintain availability during instance failures or maintenance events.

Executing the Deployment and Monitoring Status

Now you can proceed with the actual deployment process, which combines the model attachment approach with the new real-time endpoint configuration.

Python

# Attach to existing training job
estimator = SKLearn.attach(TRAINING_JOB_NAME)

# Deploy as real-time endpoint
predictor = estimator.deploy(
    initial_instance_count=INSTANCE_COUNT,
    instance_type=INSTANCE_TYPE,
    endpoint_name=ENDPOINT_NAME,
    wait=False
)

# Retrieve detailed information about the specified SageMaker endpoint
endpoint_description = sagemaker_session.sagemaker_client.describe_endpoint(EndpointName=ENDPOINT_NAME)

# Extract the current status of the endpoint from the response
status = endpoint_description['EndpointStatus']

# Display the endpoint's status
print(f"Endpoint status: {status}")

# Attach to existing training job
estimator = SKLearn.attach(TRAINING_JOB_NAME)

# Deploy as real-time endpoint
predictor = estimator.deploy(
    initial_instance_count=INSTANCE_COUNT,
    instance_type=INSTANCE_TYPE,
    endpoint_name=ENDPOINT_NAME,
    wait=False
)

# Retrieve detailed information about the specified SageMaker endpoint
endpoint_description = sagemaker_session.sagemaker_client.describe_endpoint(EndpointName=ENDPOINT_NAME)

# Extract the current status of the endpoint from the response
status = endpoint_description['EndpointStatus']

# Display the endpoint's status
print(f"Endpoint status: {status}")

The estimator.deploy() method for real-time endpoints uses different parameters compared to the serverless deployments you've worked with previously. Instead of providing a serverless_inference_config, you specify initial_instance_count and instance_type to define the persistent infrastructure. The deployment process takes significantly longer than serverless endpoints—typically 5-10 minutes versus under a minute—as SageMaker must provision dedicated EC2 instances, load your model, and perform health checks before marking the endpoint as "InService."

Introduction & Overview

Understanding Real-Time Endpoints Architecture

Join the 1M+ learners on CodeSignal

Finding and Preparing the Trained Model

Configuring Real-Time Endpoint Resources

Executing the Deployment and Monitoring Status

Considerations for Real-Time Endpoint Usage

Summary & Transition to Practice