Retrieving and Evaluating Trained Models

Welcome back! In the previous lessons, you learned how to set up your SageMaker environment, upload your data to S3, and launch a training job using the SKLearn estimator. You also saw how SageMaker manages the training process and saves your trained model artifacts to S3. These are essential skills for any machine learning workflow in the cloud.

In this lesson, you will learn how to retrieve the most recent completed SageMaker training job, download the corresponding trained model from S3, and evaluate its performance on test data. You will find your latest training job, access the model artifacts, load the trained model, and test how well it performs on new data. This is a crucial step in any machine learning project, as it allows you to verify that your model is working as expected before moving on to deployment or further experimentation.

After running training jobs in SageMaker, you may want to access a specific job to download and evaluate its model. If you already know the exact training job name you want to work with (perhaps you noted it down when the job completed, or you're working with a specific job ID from your team), you can simply set it as a variable:

# If you know the specific training job name
TRAINING_JOB_NAME = "sagemaker-scikit-learn-2025-07-22-08-42-55-894"

However, if you're working interactively and want to evaluate your most recent experiment, or if you're building an automated pipeline that needs to find the latest successful training job, you'll need to query SageMaker for this information. SageMaker provides a way to list all your training jobs and filter them by status.

To retrieve the most recent completed job programmatically, you use the list_training_jobs method. This method queries the SageMaker service and returns information about your training jobs. Let's break down how it works:

import sagemaker

# Create a SageMaker session
sagemaker_session = sagemaker.Session()

# List most recent completed training job
training_jobs = sagemaker_session.sagemaker_client.list_training_jobs(
    SortBy='CreationTime',                 # Sort jobs by creation time
    SortOrder='Descending',                # Newest jobs first
    StatusEquals='Completed',              # Only include completed jobs
    NameContains='sagemaker-scikit-learn'  # Filter to jobs containing 'sagemaker-scikit-learn' in name
)

Let's understand each parameter:

• SortBy='CreationTime': This tells SageMaker to sort the jobs by when they were created
• SortOrder='Descending': This means the newest jobs will appear first (descending from newest to oldest)
• StatusEquals='Completed': This filters out any jobs that failed, stopped, or are still running - we only want successfully completed jobs
• NameContains='sagemaker-scikit-learn': This filters the results to only include jobs whose names contain the string 'sagemaker-scikit-learn', which helps narrow down to the specific SKLearn estimator training jobs you're interested in.

The list_training_jobs method returns a dictionary containing information about the training jobs. Once you have this response, you can extract the training job name:

# Extract the name of the latest training job of the list
TRAINING_JOB_NAME = training_jobs['TrainingJobSummaries'][0]['TrainingJobName']

This line navigates through the response structure:

• training_jobs['TrainingJobSummaries'] gives us a list of training job summaries
• [0] selects the first job in our list (which is the most recent due to our descending sort order)
• ['TrainingJobName'] extracts the name field from that job summary

This approach is particularly useful when you're iterating on models and always want to evaluate the most recent successful run. The training job name is essential for the next steps, as it allows you to access the model artifacts associated with that specific job.