Evaluating a Model with PyTorch

Hello! In today's lesson, we will be diving into evaluating models in PyTorch. Evaluating the performance of a model plays a key role in the process of building an effective machine learning model. It helps us to understand the ability of the model to generalize on unseen data. We will attain this by using a test dataset, making predictions using our trained model, and comparing these predictions with the actual truth values in the test dataset.

Before evaluating the model, let's quickly recap the training process:

import torch
import torch.nn as nn
import torch.optim as optim

# Training Features
X_train = torch.tensor([
    [3.0, 0.5], [1.0, 1.0], [0.5, 2.0], [2.0, 1.5],
    [3.5, 3.0], [2.0, 2.5], [1.5, 1.0], [0.5, 0.5],
    [2.5, 0.8], [2.1, 2.0], [1.2, 0.5], [0.7, 1.5]
], dtype=torch.float32)

# Training Targets
y_train = torch.tensor([[1], [0], [0], [1], [1], [0], [1], [0], [1], [0], [1], [0]], dtype=torch.float32)

# Define the model using nn.Sequential
model = nn.Sequential(
    nn.Linear(2, 10),
    nn.ReLU(),
    nn.Linear(10, 1),
    nn.Sigmoid()
)

# Define loss function and optimizer
criterion = nn.BCELoss()
optimizer = optim.Adam(model.parameters(), lr=0.01)  

# Train the model for 50 epochs
for epoch in range(50):  
    model.train()  # Set the model to training mode
    optimizer.zero_grad()  # Zero the gradients
    outputs = model(X_train)  # Compute predictions
    loss = criterion(outputs, y_train)  # Compute the loss
    loss.backward()  # Compute the gradient
    optimizer.step()  # Update the parameters

Here’s a summary of the steps:

1. Data Preparation: Defined training features X_train and targets y_train.
2. Model Architecture: Created a network using nn.Sequential with one hidden layer (ReLU) and an output layer (Sigmoid).
3. Loss and Optimizer: Used Binary Cross-Entropy Loss (BCELoss) and the Adam optimizer.
4. Training Loop: Trained for 50 epochs, performing forward pass, loss calculation, backpropagation, and parameter updates in each epoch.

Now let's move to evaluating our model.

Before we evaluate our model, we need to prepare our test dataset. The test dataset consists of new data points that the model has never seen before. This helps us understand how well our model generalizes to unseen data.

Let's define our testing data with the same format our model was trained on in the previous lessons:

import torch

# Test Features
X_test = torch.tensor([[2.5, 1.0], [0.8, 0.8], [1.0, 2.0], [3.0, 2.5]], dtype=torch.float32)
# Test Targets
y_test = torch.tensor([[1], [0], [0], [1]], dtype=torch.float32)

With our test data ready, we can now move on to evaluating our model's performance using these new examples.