Evaluating a predictive model's performance is crucial in machine learning. To understand how well our model predicts outcomes, we need effective metrics. In this lesson, we will explore three important metrics used in regression analysis: Mean Squared Error (MSE), Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE). By the end of this lesson, you will know what these metrics are, how to calculate them, and understand their differences. Let's get started!

Before we dive into the specifics, let's briefly touch on regression metrics and their importance. These metrics help us quantify the difference between actual outcomes (true values) and model predictions. This allows us to assess our model's performance. The example code below shows how to calculate MSE, MAE, and RMSE.

from sklearn.metrics import mean_squared_error, mean_absolute_error
import numpy as np

# Sample regression dataset
y_true = np.array([3.0, -0.5, 2.0, 7.0])
y_pred = np.array([2.5, 0.0, 2.0, 8.0])

# Calculating MSE, MAE, and RMSE
mse = mean_squared_error(y_true, y_pred)
mae = mean_absolute_error(y_true, y_pred)
rmse = np.sqrt(mean_squared_error(y_true, y_pred))

print(f"MSE: {mse}, MAE: {mae}, RMSE: {rmse}")
# Output: MSE: 0.375, MAE: 0.5, RMSE: 0.6123724356957945

This code calculates the three metrics for a set of true and predicted values. Let’s delve into each metric to understand their implications.

MSE is calculated by summing squared differences between true and predicted values, then dividing by the number of samples. We'll use the already-implemented mean_squared_error function from the sklearn to calculate it:

from sklearn.metrics import mean_squared_error
import numpy as np

# True and predicted values
y_true = [3.0, -0.5, 2.0, 7.0]
y_pred = [2.5, 0.0, 2.0, 8.0]

# Mean Squared Error
mse = mean_squared_error(y_true, y_pred)
print(f"MSE: {mse}")  # Output: MSE: 0.375