Deploying Locally Trained Models to SageMaker

Deploying Locally Trained Models to SageMaker | CodeSignal Learn

Introduction & Lesson Overview

Welcome to your first lesson on deploying models with SageMaker! You've already built a solid foundation by training machine learning models both locally and in the cloud using SageMaker. Now it's time to take the next crucial step in your machine learning journey: making your models available for real-world predictions through deployment. In this lesson, you'll learn how to deploy a model that you trained on your local machine to a SageMaker endpoint. This is an excellent starting point because you can use the familiar models you've already created while learning the essential deployment concepts that apply to all SageMaker deployments. By the end of this lesson, you'll understand how to package your local model artifacts, upload them to Amazon S3, configure a SageMaker deployment environment, and create a working endpoint that can serve predictions. This foundational knowledge will prepare you for more advanced deployment scenarios later in the course.

Understanding SageMaker Endpoints

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Packaging Your Local Model for Deployment

To deploy your locally trained model to SageMaker, you need to package it in a format that SageMaker understands. SageMaker expects model artifacts to be stored in a compressed tar.gz file, which is essentially a zipped archive containing your trained model files. Let's start by creating the model package: Python import tarfile # Create model artifacts tar.gz file (SageMaker format) with tarfile.open('model.tar.gz', 'w:gz') as tar: tar.add('trained_model.joblib') import tarfile # Create model artifacts tar.gz file (SageMaker format) with tarfile.open('model.tar.gz', 'w:gz') as tar: tar.add('trained_model.joblib') The key part here is the tarfile.open('model.tar.gz', 'w:gz') line, which creates a new compressed archive file. The'w:gz' parameter tells Python to create a writable, gzip-compressed tar file. Then, tar.add('trained_model.joblib') adds your trained model file to the archive. The important thing to understand is that whatever files your model needs to make predictions must be included in this tar.gz archive. If your model depends on additional files like preprocessing pipelines, feature encoders, or configuration files, you would add those to the archive as well. The resulting model.tar.gz file is now in the exact format that SageMaker expects. This standardized packaging approach ensures that SageMaker can properly extract and load your model artifacts during deployment, regardless of which machine learning framework you used for training.

Uploading Model Artifacts to Amazon S3

Once your model is packaged in the correct format, the next step is to upload it to Amazon S3, where SageMaker can access it during deployment: Pythonimport sagemaker # Create a SageMaker session sagemaker_session = sagemaker.Session() # Get the default bucket default_bucket = sagemaker_session.default_bucket() # Upload model to S3 model_artifact_uri = sagemaker_session.upload_data( path='model.tar.gz', bucket=default_bucket, key_prefix='models/local-trained' ) # Print model artifacts URI print(f"Model artifacts uploaded to: {model_artifact_uri}")import sagemaker # Create a SageMaker session sagemaker_session = sagemaker.Session() # Get the default bucket default_bucket = sagemaker_session.default_bucket() # Upload model to S3 model_artifact_uri = sagemaker_session.upload_data( path='model.tar.gz', bucket=default_bucket, key_prefix='models/local-trained' ) # Print model artifacts URI print(f"Model artifacts uploaded to: {model_artifact_uri}") The sagemaker_session.upload_data() method handles all the complexity of uploading your file to S3. Let's break down the parameters: path='model.tar.gz' specifies the local file you want to upload bucket=default_bucket tells SageMaker to use your account's default SageMaker bucket key_prefix='models/local-trained' creates an organized folder structure in S3 where your model will be stored When you run this code, you'll see output similar to this: textModel artifacts uploaded to: s3://sagemaker-us-east-1-123456789012/models/local-trained/model.tar.gzModel artifacts uploaded to: s3://sagemaker-us-east-1-123456789012/models/local-trained/model.tar.gz This S3 URI is crucial because you'll need it in the next step to tell SageMaker where to find your model artifacts. The URI follows the format s3://bucket-name/key-prefix/filename, and SageMaker will use this exact location to download and extract your model during deployment.

Setting Up Configuration Variables

Now that your model artifacts are stored in S3, you need to configure the SageMaker deployment environment. Let's start by setting up the necessary configuration variables: Python# Get account id from the current AWS session account_id = sagemaker_session.boto_session.client('sts').get_caller_identity()['Account'] # Define the SageMaker execution role ARN using the account id SAGEMAKER_ROLE = f"arn:aws:iam::{account_id}:role/SageMakerDefaultExecution" # Name for the SageMaker endpoint to be created ENDPOINT_NAME = "california-housing-local-model"# Get account id from the current AWS session account_id = sagemaker_session.boto_session.client('sts').get_caller_identity()['Account'] # Define the SageMaker execution role ARN using the account id SAGEMAKER_ROLE = f"arn:aws:iam::{account_id}:role/SageMakerDefaultExecution" # Name for the SageMaker endpoint to be created ENDPOINT_NAME = "california-housing-local-model" The account_id is retrieved automatically from your AWS session and is used to construct the SAGEMAKER_ROLE ARN, which grants SageMaker the necessary permissions for deployment. The ENDPOINT_NAME variable specifies the name of your SageMaker endpoint—choose a descriptive and unique name within your AWS account. If you deploy multiple models, simply change this value to avoid naming conflicts.

Creating the Entry Point Script

Before we can deploy our model, we need to create an entry point script that tells SageMaker how to load and use your model for predictions. This script acts as the bridge between SageMaker's inference infrastructure and your specific model. Create a file called entry_point.py with the following content: Python import os import joblib def model_fn(model_dir): """Load model for inference""" model = joblib.load(os.path.join(model_dir, 'trained_model.joblib')) return model import os import joblib def model_fn(model_dir): """Load model for inference""" model = joblib.load(os.path.join(model_dir, 'trained_model.joblib')) return model The model_fn() function is a SageMaker convention that defines how to load your model from the extracted tar.gz archive. When SageMaker receives a prediction request, it automatically extracts your model artifacts to the model_dir directory and calls this function to load the model into memory. The function should return the loaded model object that will be used for making predictions. This entry point script is essential because SageMaker needs to know exactly how to initialize your model. Different machine learning frameworks and model types may require different loading procedures, so this script gives you complete control over the model loading process.

Creating the SageMaker Model Object

With our entry point script ready, we can now create the SageMaker model object that combines your model artifacts with the inference logic. This object serves as a blueprint that tells SageMaker everything it needs to know about your model: Python from sagemaker.sklearn.model import SKLearnModel # Create a SKLearnModel from local model artifacts model = SKLearnModel(model_data=model_artifact_uri, # S3 location of model.tar.gz role=SAGEMAKER_ROLE, # IAM role for SageMaker entry_point='entry_point.py', # Inference script framework_version='1.2-1', # scikit-learn version py_version='py3', # Python version sagemaker_session=sagemaker_session # SageMaker session) from sagemaker.sklearn.model import SKLearnModel # Create a SKLearnModel from local model artifacts model = SKLearnModel(model_data=model_artifact_uri, # S3 location of model.tar.gz role=SAGEMAKER_ROLE, # IAM role for SageMaker entry_point='entry_point.py', # Inference script framework_version='1.2-1', # scikit-learn version py_version='py3', # Python version sagemaker_session=sagemaker_session # SageMaker session) The SKLearnModel class is specifically designed for scikit-learn models. The model_data parameter points to your S3 model artifacts, while entry_point='entry_point.py' specifies the Python script we just created that defines how to load and use your model for predictions. The framework_version and py_version parameters ensure that SageMaker uses the correct software environment to run your model. This model object is now ready for deployment, but first we need to configure the serverless inference settings that will determine how much computing power your endpoint will have available.

Configuring Serverless Inference Settings

Now that we have our model object configured, we need to specify the resource requirements for our serverless endpoint. These settings determine how much memory your model will have access to and how many simultaneous requests it can handle: Python from sagemaker.serverless import ServerlessInferenceConfig # Configure serverless inference serverless_config = ServerlessInferenceConfig(memory_size_in_mb=2048, max_concurrency=10) from sagemaker.serverless import ServerlessInferenceConfig # Configure serverless inference serverless_config = ServerlessInferenceConfig(memory_size_in_mb=2048, max_concurrency=10) The ServerlessInferenceConfig allows you to specify resource requirements for your endpoint. The memory_size_in_mb=2048 allocates 2 GB of memory, which is typically sufficient for most scikit-learn models, and max_concurrency=10 limits the number of simultaneous inference requests to control costs and prevent overwhelming your model.

Deploying Your Model to a SageMaker Serverless Endpoint

With your model configured, you're ready to deploy it to a live SageMaker endpoint. The deployment process creates the necessary AWS infrastructure and makes your model available for real-time predictions: Python# Deploy model as serverless endpoint asynchronously predictor = model.deploy( serverless_inference_config=serverless_config, endpoint_name=ENDPOINT_NAME, wait=False ) # Retrieve detailed information about the specified SageMaker endpoint endpoint_description = sagemaker_session.sagemaker_client.describe_endpoint(EndpointName=ENDPOINT_NAME) # Extract the current status of the endpoint from the response status = endpoint_description['EndpointStatus'] # Display the endpoint's status print(f"Endpoint status: {status}")# Deploy model as serverless endpoint asynchronously predictor = model.deploy( serverless_inference_config=serverless_config, endpoint_name=ENDPOINT_NAME, wait=False ) # Retrieve detailed information about the specified SageMaker endpoint endpoint_description = sagemaker_session.sagemaker_client.describe_endpoint(EndpointName=ENDPOINT_NAME) # Extract the current status of the endpoint from the response status = endpoint_description['EndpointStatus'] # Display the endpoint's status print(f"Endpoint status: {status}") The model.deploy() method creates your live endpoint infrastructure. We set wait=False to deploy asynchronously, which means the method returns immediately instead of blocking your code for several minutes while AWS provisions the resources. Since the deployment happens in the background, we need to actively check its progress. The describe_endpoint() call shows us the current deployment status. Initially, you'll see: textEndpoint status: CreatingEndpoint status: Creating Your endpoint moves through a predictable lifecycle: it starts as Creating while AWS sets up the infrastructure, then transitions to InService when ready for predictions, or Failed if something goes wrong. The deployment typically takes 2-5 minutes for serverless endpoints. If you prefer to wait automatically, you can set wait=True in the deploy() method and your code will block until the endpoint is fully deployed. Alternatively, you can periodically run the describe_endpoint() code above to manually check the status. Once you see InService, your model is live and ready to handle requests. Until then, you'll need to wait for the provisioning process to complete.

Connecting to Your SageMaker Endpoint

Once your endpoint shows an InService status, it's ready to handle prediction requests. But what exactly is this endpoint? Your SageMaker endpoint is essentially a REST API - a web service running on AWS servers that accepts HTTP requests and returns HTTP responses. Just like when you visit a website, your code will send HTTP requests to a specific URL, and the endpoint will respond with predictions. This is where SageMaker's Predictor class becomes essential. Rather than forcing you to construct HTTP requests manually, it provides a simple Python interface that handles all the networking complexity behind the scenes. Python from sagemaker.predictor import Predictor # Connect to the deployed SageMaker endpoint for inference predictor = Predictor(endpoint_name=ENDPOINT_NAME) from sagemaker.predictor import Predictor # Connect to the deployed SageMaker endpoint for inference predictor = Predictor(endpoint_name=ENDPOINT_NAME) When you create this Predictor object, it automatically discovers your endpoint's URL using the endpoint name, manages authentication with your AWS credentials, and prepares to handle the HTTP communication. Think of it as a specialized HTTP client that knows exactly how to talk to SageMaker endpoints - you get all the power of REST API communication without writing a single line of HTTP code. However, creating the connection is only half the battle. We still need to solve a fundamental data transfer problem.

Configuring Data Serialization

Evaluating Model Performance

With our predictor properly configured to handle the data conversion and HTTP communication, we can now send real data to test the deployed model's performance. This evaluation serves a critical purpose: verifying that your model works correctly in the production environment and maintains the same accuracy you observed during local development. Pythonimport pandas as pd import numpy as np from sklearn.metrics import mean_squared_error, r2_score # Path to local test dataset TEST_DATA_FILE = "data/california_housing_test.csv" # Load and split test data into features and target variable test_df = pd.read_csv(TEST_DATA_FILE) X_test = test_df.drop("MedHouseVal", axis=1) y_test = test_df["MedHouseVal"] # Make predictions on entire test set predictions = predictor.predict(X_test.values) # Evaluate the model test_r2 = r2_score(y_test, predictions) test_rmse = np.sqrt(mean_squared_error(y_test, predictions)) # Print the results print(f"R² Score: {test_r2:.4f}") print(f"RMSE: {test_rmse:.4f}")import pandas as pd import numpy as np from sklearn.metrics import mean_squared_error, r2_score # Path to local test dataset TEST_DATA_FILE = "data/california_housing_test.csv" # Load and split test data into features and target variable test_df = pd.read_csv(TEST_DATA_FILE) X_test = test_df.drop("MedHouseVal", axis=1) y_test = test_df["MedHouseVal"] # Make predictions on entire test set predictions = predictor.predict(X_test.values) # Evaluate the model test_r2 = r2_score(y_test, predictions) test_rmse = np.sqrt(mean_squared_error(y_test, predictions)) # Print the results print(f"R² Score: {test_r2:.4f}") print(f"RMSE: {test_rmse:.4f}") We begin by loading our test dataset and separating the input features from the target values we want to predict. The crucial step happens with predictor.predict(X_test.values) - this call triggers the entire serialization process we just described, converting your data to CSV, sending it to the SageMaker endpoint, and receiving the model's predictions back. The evaluation metrics provide the final validation of your successful deployment. Typical results might look like: textR² Score: 0.6456 RMSE: 0.6814R² Score: 0.6456 RMSE: 0.6814 When these metrics align with your local results, you can be confident that your deployment process preserved your model's predictive capabilities and that your endpoint is ready for real-world use.

Deploying Locally Trained Models to SageMaker

Introduction & Lesson Overview

Understanding SageMaker Endpoints

Join the 1M+ learners on CodeSignal

Packaging Your Local Model for Deployment

Uploading Model Artifacts to Amazon S3

Setting Up Configuration Variables

Creating the Entry Point Script

Creating the SageMaker Model Object

Configuring Serverless Inference Settings

Deploying Your Model to a SageMaker Serverless Endpoint

Connecting to Your SageMaker Endpoint

Configuring Data Serialization

Evaluating Model Performance

Summary & Next Steps