Welcome! Today, we're learning to evaluate your machine learning model's performance. Evaluating your model is crucial because it tells you how well it will predict new data it hasn't seen before. In simpler terms, it tells you if your model is good at its job.

We will focus on one metric – Mean Squared Error (MSE). This metric is like a report card for your model, showing its prediction accuracy. By the end of this lesson, you'll know how to calculate it and understand what it mean.s

Let's review what we've done so far. We have been working with synthetic data representing house areas and their prices. We used this data to train a simple linear regression model to predict house prices based on their area. Here's a reminder snippet:

With our model trained, we can evaluate its performance.

Mean Squared Error (MSE) measures how far off our model's predictions are from the actual values. It’s like checking how precise your aim is in darts. The lower the MSE, the better.

Steps to calculate MSE:

1. Make predictions using your model.
2. Calculate the difference between actual and predicted prices for each house.
3. Square these differences.
4. Find the average of these squared differences.

Mathematically, it looks like this: $\text{MSE} = \frac{1}{N} \sum_{i=1}^{N} (y_i - \hat{y}_i)^2$ where:

• $N$ is the number of data points,
• $y_i$ is the actual value for the $i$-th data point,
• $\hat{y}_i$ is the predicted value for the $i$-th data point.

Let's visualize it with a plot:

Here, green vertical lines show distance between the actual price and the model's prediction. If we square all these distances and find their average, it will be the MSE metric!

Here's the code:

This will output something like:

In real life, MSE helps you understand if your model's predictions are close to the actual prices. For example, if predicting toy prices, an MSE of 1000 means predictions are off by about 31.62 (since $\sqrt{1000} \approx 31.62$).

So, what do our MSE score tell us? Lower values are better. If your MSE is high, it means your model’s predictions are not accurate. For example, while predicting toy prices, an MSE of 10 might be great, but an MSE of 1000 means the model is often very wrong.

Understanding these metrics helps you improve your model. Suppose your MSE score is high; to improve your predictions, you might need to consider other features or preprocess data more accurately or even choose a different model!

Today, you learned to evaluate your model's performance using Mean Squared Error (MSE). MSE measures how close your model’s predictions are to actual values.

Understanding these metrics helps you assess and improve your model. Next, we'll practice using MSE Score to evaluate models on CodeSignal!