Evaluating Your Model's Performance

Welcome! Today, we're learning to evaluate your machine learning model's performance. Evaluating your model is crucial because it tells you how well it will predict new data it hasn't seen before. In simpler terms, it tells you if your model is good at its job.

We will focus on one metric – Mean Squared Error (MSE). This metric is like a report card for your model, showing its prediction accuracy. By the end of this lesson, you'll know how to calculate it and understand what it mean.s

Let's review what we've done so far. We have been working with synthetic data representing house areas and their prices. We used this data to train a simple linear regression model to predict house prices based on their area. Here's a reminder snippet:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression

# Generate synthetic data
np.random.seed(42)
num_samples = 100
area = np.random.uniform(500, 3500, num_samples)  # House area in square feet
base_price = 50000
price_per_sqft = 200
noise = np.random.normal(0, 25000, num_samples)  # Adding some noise
price = base_price + (area * price_per_sqft) + noise

# Create DataFrame
df = pd.DataFrame({'Area': area, 'Price': price})

# Extract features and target variable
X = df['Area'].values.reshape(-1, 1)
y = df['Price'].values

# Initialize and train the model
model = LinearRegression()
model.fit(X, y)

With our model trained, we can evaluate its performance.

Mean Squared Error (MSE) measures how far off our model's predictions are from the actual values. It’s like checking how precise your aim is in darts. The lower the MSE, the better.

Steps to calculate MSE:

1. Make predictions using your model.
2. Calculate the difference between actual and predicted prices for each house.
3. Square these differences.
4. Find the average of these squared differences.

Mathematically, it looks like this: $\text{MSE} = \frac{1}{N} \sum_{i=1}^{N} (y_i - \hat{y}_i)^2$ where:

• $N$ is the number of data points,
• $y_i$ is the actual value for the $i$-th data point,
• $\hat{y}_i$ is the predicted value for the $i$-th data point.

Let's visualize it with a plot:

Here, green vertical lines show distance between the actual price and the model's prediction. If we square all these distances and find their average, it will be the MSE metric!