Welcome to another pivotal moment in your SageMaker deployment journey! Over the past three lessons, you've mastered serverless deployments, learning how to deploy various model types to cost-effective endpoints that scale to zero and charge only for actual usage.

Now you're ready to explore a fundamentally different paradigm: real-time endpoints that maintain persistent, always-on infrastructure for consistent, low latency responses.

By the end of this lesson, you'll understand how to configure persistent endpoints, select appropriate instance types, test deployments with production-like workflows, and monitor endpoint performance—completing your understanding of SageMaker's deployment options.

Real-time endpoints represent SageMaker's solution for applications that require persistent, high-performance inference infrastructure. Unlike the serverless endpoints you've worked with in previous lessons, real-time endpoints maintain dedicated compute instances that remain active and ready to serve predictions at all times. This architectural approach eliminates the cold start delays that can occur with serverless endpoints when they scale from zero, ensuring that your model can respond to prediction requests with consistent, predictable latency.

The key architectural difference lies in the infrastructure persistence model. When you deploy a real-time endpoint, SageMaker provisions dedicated EC2 instances that host your model continuously. These instances run your model in memory, maintaining the loaded state and ready-to-serve configuration that enables immediate response to incoming requests. This persistent architecture makes real-time endpoints particularly well-suited for applications that require guaranteed response times, handle steady streams of prediction requests, or serve as critical components in time-sensitive decision-making processes.

Real-time endpoints also provide sophisticated auto-scaling capabilities that allow them to automatically adjust the number of running instances based on incoming traffic patterns. When request volume increases, SageMaker can automatically launch additional instances to maintain performance levels. When traffic decreases, the service can scale down while maintaining a minimum number of instances to ensure availability. This auto-scaling behavior provides the flexibility to handle traffic spikes while maintaining cost efficiency during lower-demand periods.

The use cases for real-time endpoints span across industries and applications where immediate responses are critical. Financial services use real-time endpoints for fraud detection systems that must evaluate transactions within milliseconds. E-commerce platforms deploy recommendation engines that provide instant product suggestions as users browse. Healthcare applications rely on real-time endpoints for diagnostic assistance tools that support clinical decision-making. Manufacturing systems use real-time endpoints for quality control processes that must evaluate products as they move through production lines. In each of these scenarios, the consistent low latency and high availability provided by real-time endpoints are essential for the application's success.

The deployment process for real-time endpoints builds upon the concepts you've learned in previous lessons while introducing new configuration options specific to persistent infrastructure. As you've done before, we'll start by establishing a SageMaker session and locating a trained model, but this time you'll configure the deployment for always-on availability rather than serverless scaling.

This initial setup should look familiar from your previous lessons, as you're using the same approach to locate your most recent completed training job. The NameContains='sagemaker-scikit-learn' filter helps you identify estimator-based training jobs, which are ideal for demonstrating real-time endpoint deployment because they can be easily attached using the SKLearn.attach() method you learned before.

With your training job identified, you're ready to move on to configuring the persistent infrastructure that will distinguish this deployment from the serverless endpoints you've worked with previously.

The real-time endpoint configuration introduces new parameters that define the persistent infrastructure characteristics. Unlike serverless endpoints that automatically manage scaling and resource allocation, real-time endpoints require you to explicitly specify the compute resources that will host your model.

These configuration parameters control the fundamental characteristics of your real-time endpoint infrastructure:

• INSTANCE_TYPE: Specifies the EC2 instance type that will host your model. The ml.m5.large instance type provides a robust combination of compute, memory, and network performance that's well-suited for production-grade machine learning inference. This instance type offers 2 vCPUs and 8 GB of memory, which provides ample resources for scikit-learn models while ensuring consistent performance under load. The M5 family is optimized for general-purpose computing with balanced resources, making it an excellent choice for a wide range of machine learning workloads.

• INSTANCE_COUNT: Determines how many instances will be deployed to serve your model. Since real-time endpoints charge for all running instances continuously (unlike serverless endpoints that only charge for actual usage), each additional instance directly multiplies your hourly costs. Starting with a single instance minimizes costs during development and testing phases while still providing the persistent availability that real-time endpoints offer. Production deployments often use multiple instances to provide redundancy and handle higher request volumes, but this comes with proportionally higher costs as SageMaker automatically distributes incoming requests across all available instances.

Understanding these parameters is crucial for optimizing both performance and cost in your real-time endpoint deployments. The instance type selection directly impacts your model's response time and throughput capabilities, while the instance count affects your endpoint's ability to handle concurrent requests and maintain availability during instance failures or maintenance events.

Now you can proceed with the actual deployment process, which combines the model attachment approach with the new real-time endpoint configuration.

The estimator.deploy() method for real-time endpoints uses different parameters compared to the serverless deployments you've worked with previously. Instead of providing a serverless_inference_config, you specify initial_instance_count and instance_type to define the persistent infrastructure. The deployment process takes significantly longer than serverless endpoints—typically 5-10 minutes versus under a minute—as SageMaker must provision dedicated EC2 instances, load your model, and perform health checks before marking the endpoint as "InService."

Before deploying real-time endpoints in production, you must understand the key trade-offs that will impact both performance and costs. These considerations help determine when real-time endpoints are the optimal choice for your specific use case.

• Cost implications: Real-time endpoints incur continuous costs for all provisioned instances regardless of usage, unlike serverless endpoints that only charge for actual requests. This makes them most economical for steady, predictable traffic patterns.

• Instance selection and sizing: Balance your model's resource requirements against costs. Instances like ml.m5.large provide a good balance of performance and cost for many workloads, while smaller instances like ml.t3.medium may be more economical for lightweight models. Larger instances like ml.m5.xlarge or ml.c5.2xlarge provide more power at proportionally higher costs. GPU-enabled instances (ml.p3 or ml.g4dn families) may be necessary for deep learning models but significantly increase costs.

• Scaling and availability: Production deployments typically need multiple instances for redundancy and traffic handling. Auto-scaling takes time as new instances must launch and initialize, requiring proactive rather than reactive scaling policies.

• Monitoring and maintenance: Real-time endpoints require continuous monitoring of instance health, performance metrics, and scaling behavior. Model updates need careful coordination across running instances to minimize downtime.

These considerations help you choose between serverless and real-time endpoints based on your traffic patterns, latency requirements, and budget constraints rather than technical preferences alone.

Congratulations! You've now mastered real-time endpoint deployment in SageMaker, completing your understanding of both serverless and persistent deployment patterns. You can now create always-on inference infrastructure, configure instance types for production workloads, and choose the right deployment type based on your application's specific requirements.

Let's dive into the hands-on practice that will cement your understanding of high-throughput, production-ready model deployment!