Building a Basic RNN Model with PyTorch

Introduction to Building a Basic RNN Model

Welcome to the next step in your journey of mastering Recurrent Neural Networks (RNNs) for time series analysis. In the previous lesson, you learned how to prepare time series data for RNNs by normalizing it and converting it into sequences. This foundation is crucial as we now move on to building and evaluating a basic RNN model. In this lesson, you will learn how to define, train, and evaluate a simple RNN model using PyTorch . By the end of this lesson, you will be able to implement a basic RNN model to predict time series values and assess its performance.

Defining the RNN Model

Let's start by defining a basic RNN model. We will use PyTorch, which is a powerful tool for building neural networks. The model will consist of an RNN layer, followed by a Linear layer. The RNN layer is responsible for processing the sequences of data, while the Linear layer outputs the prediction. Here's how you can define the model: Python import torch import torch.nn as nn class SimpleRNNModel(nn.Module): def __init__(self, input_size, hidden_size, output_size): super(SimpleRNNModel, self).__init__() self.rnn = nn.RNN(input_size, hidden_size, batch_first=True) self.linear = nn.Linear(hidden_size, output_size) def forward(self, x): out, _ = self.rnn(x) out = self.linear(out[:, -1, :]) # Take the last output of the sequence return out # Define RNN model input_size = 1 hidden_size = 10 output_size = 1 model = SimpleRNNModel(input_size, hidden_size, output_size) In this code, we define a class SimpleRNNModel that subclasses nn.Module . The model consists of an RNN layer with 10 hidden units and a Linear layer to produce the final output. The forward method defines the forward pass of the model, where we take the last output of the RNN sequence to pass through the Linear layer.

Splitting the Data

For time series data, it's important to split the data chronologically to avoid data leakage. This means using the earlier part of the series for training and the later part for testing. Here's how you can split the data: Python # Determine split index split_idx = int(len(X) * 0.8) # Chronological split X_train, X_test = X[:split_idx], X[split_idx:] y_train, y_test = y[:split_idx], y[split_idx:] In this code, we use the first 80% of the data for training and the remaining 20% for testing, preserving the temporal order of the time series. This approach ensures that the model is always tested on data points that occur after those it was trained on, which is essential for time series forecasting.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Training the RNN Model

With the model defined and data split, the next step is to train it using the training dataset. Training involves feeding the model with input sequences and adjusting its parameters to minimize the loss function.

Here's how you can train the model:

Python

import torch.optim as optim

# Define loss function and optimizer
criterion = nn.MSELoss()  # Mean Squared Error loss for regression tasks
optimizer = optim.Adam(model.parameters(), lr=0.001)  # Adam optimizer for updating model weights

# Training loop parameters
epochs = 25
batch_size = 16

# List to store loss values for each epoch (for plotting later)
losses = []

for epoch in range(epochs):
    model.train()  # Set the model to training mode
    epoch_loss = 0  # Track loss for this epoch

    # Loop over the training data in batches
    for i in range(0, len(X_train), batch_size):
        # Prepare batch data as PyTorch tensors
        X_batch = torch.tensor(X_train[i:i+batch_size], dtype=torch.float32)
        y_batch = torch.tensor(y_train[i:i+batch_size], dtype=torch.float32)

        # ---- Forward Pass ----
        # Pass the input batch through the model to get predictions
        outputs = model(X_batch)

        # Compute the loss between predictions and actual values
        loss = criterion(outputs, y_batch)

        # ---- Backpropagation ----
        # Zero the gradients from the previous step
        optimizer.zero_grad()
        # Compute gradients of the loss with respect to model parameters
        loss.backward()
        # Update model parameters using the optimizer
        optimizer.step()

        # Accumulate loss for this batch
        epoch_loss += loss.item()

    # Calculate average loss for the epoch and store it
    avg_loss = epoch_loss / (len(X_train) // batch_size)
    losses.append(avg_loss)
    print(f'Epoch [{epoch+1}/{epochs}], Loss: {avg_loss:.4f}')

import torch.optim as optim

# Define loss function and optimizer
criterion = nn.MSELoss()  # Mean Squared Error loss for regression tasks
optimizer = optim.Adam(model.parameters(), lr=0.001)  # Adam optimizer for updating model weights

# Training loop parameters
epochs = 25
batch_size = 16

# List to store loss values for each epoch (for plotting later)
losses = []

for epoch in range(epochs):
    model.train()  # Set the model to training mode
    epoch_loss = 0  # Track loss for this epoch

    # Loop over the training data in batches
    for i in range(0, len(X_train), batch_size):
        # Prepare batch data as PyTorch tensors
        X_batch = torch.tensor(X_train[i:i+batch_size], dtype=torch.float32)
        y_batch = torch.tensor(y_train[i:i+batch_size], dtype=torch.float32)

        # ---- Forward Pass ----
        # Pass the input batch through the model to get predictions
        outputs = model(X_batch)

        # Compute the loss between predictions and actual values
        loss = criterion(outputs, y_batch)

        # ---- Backpropagation ----
        # Zero the gradients from the previous step
        optimizer.zero_grad()
        # Compute gradients of the loss with respect to model parameters
        loss.backward()
        # Update model parameters using the optimizer
        optimizer.step()

        # Accumulate loss for this batch
        epoch_loss += loss.item()

    # Calculate average loss for the epoch and store it
    avg_loss = epoch_loss / (len(X_train) // batch_size)
    losses.append(avg_loss)
    print(f'Epoch [{epoch+1}/{epochs}], Loss: {avg_loss:.4f}')

Explanation:

Forward Pass:
In the forward pass, the input batch (X_batch) is passed through the model (model(X_batch)). The model processes the input data and produces predictions (outputs). These predictions are then compared to the actual target values (y_batch) using the loss function (criterion). The result is a single loss value that quantifies how well the model's predictions match the actual values.
Backpropagation:
Backpropagation is the process of updating the model's parameters to minimize the loss. First, we clear any previously stored gradients with optimizer.zero_grad(). Then, loss.backward() computes the gradients of the loss with respect to each model parameter. Finally, optimizer.step() updates the model parameters using these gradients, moving them in the direction that reduces the loss.
Batch Training:
The training data is processed in small batches (here, of size 16) rather than all at once. This helps the model learn more efficiently and makes training feasible for large datasets.
Epochs:
The entire training dataset is processed multiple times (here, 25 epochs) to allow the model to learn from the data.
Loss Tracking:
The average loss for each epoch is stored in the losses list, which can be used later to plot the training loss curve and monitor the model's learning progress.

Evaluating the RNN Model

After training the model, it's important to evaluate its performance on the test set. One way to do this is by plotting the training loss curve and calculating the RMSE on the test data.

Here's how you can evaluate the model:

Python

import matplotlib.pyplot as plt
from sklearn.metrics import mean_squared_error
import numpy as np

# Evaluate: Plot loss curve
# (Assuming loss values are stored in a list `losses` during training)
plt.figure(figsize=(8, 4))
plt.plot(losses, label="Training Loss")
plt.xlabel("Epochs")
plt.ylabel("Loss (MSE)")
plt.title("Training Loss Curve")
plt.legend()
plt.show()

# Compute RMSE on test data
model.eval()
with torch.no_grad():
    X_test_tensor = torch.tensor(X_test, dtype=torch.float32)
    y_pred_test = model(X_test_tensor).numpy()

y_pred_test = np.clip(y_pred_test, 0, 1)  # Ensure predictions stay in [0,1] before inverse scaling
y_pred_test_rescaled = scaler.inverse_transform(y_pred_test)

y_test_rescaled = scaler.inverse_transform(y_test.reshape(-1, 1))  # Ensure y_test is 2D
rmse_test = np.sqrt(mean_squared_error(y_test_rescaled, y_pred_test_rescaled))
print(f"Test RMSE: {rmse_test}")

import matplotlib.pyplot as plt
from sklearn.metrics import mean_squared_error
import numpy as np

# Evaluate: Plot loss curve
# (Assuming loss values are stored in a list `losses` during training)
plt.figure(figsize=(8, 4))
plt.plot(losses, label="Training Loss")
plt.xlabel("Epochs")
plt.ylabel("Loss (MSE)")
plt.title("Training Loss Curve")
plt.legend()
plt.show()

# Compute RMSE on test data
model.eval()
with torch.no_grad():
    X_test_tensor = torch.tensor(X_test, dtype=torch.float32)
    y_pred_test = model(X_test_tensor).numpy()

y_pred_test = np.clip(y_pred_test, 0, 1)  # Ensure predictions stay in [0,1] before inverse scaling
y_pred_test_rescaled = scaler.inverse_transform(y_pred_test)

y_test_rescaled = scaler.inverse_transform(y_test.reshape(-1, 1))  # Ensure y_test is 2D
rmse_test = np.sqrt(mean_squared_error(y_test_rescaled, y_pred_test_rescaled))
print(f"Test RMSE: {rmse_test}")

In this code, we first plot the training loss curve using Matplotlib. We then compute the RMSE on the test data by predicting the values using the trained model and rescaling them back to the original scale. The RMSE is calculated using the mean_squared_error function from scikit-learn.

Making Predictions and Visualizing Results

Once the model is trained and evaluated, we can use it to make predictions on the test data. It's important to rescale the predictions back to the original scale to interpret them correctly. Visualizing the actual vs. predicted values can help assess the model's accuracy.

Here's how you can make predictions and visualize the results:

Python

# Plot actual vs. predicted values on test data
plt.figure(figsize=(10, 5))
plt.plot(y_test_rescaled, label="Actual")
plt.plot(y_pred_test_rescaled, label="Predicted", linestyle='dashed')
plt.xlabel("Time")
plt.ylabel("Passengers")
plt.title("Actual vs. Predicted Values on Test Data")
plt.legend()
plt.show()

# Plot actual vs. predicted values on test data
plt.figure(figsize=(10, 5))
plt.plot(y_test_rescaled, label="Actual")
plt.plot(y_pred_test_rescaled, label="Predicted", linestyle='dashed')
plt.xlabel("Time")
plt.ylabel("Passengers")
plt.title("Actual vs. Predicted Values on Test Data")
plt.legend()
plt.show()

In this code, we plot the actual and predicted values on the test data using Matplotlib. The y_test_rescaled and y_pred_test_rescaled variables contain the rescaled actual and predicted values, respectively. The plot provides a visual comparison of the model's predictions against the actual test data, allowing you to assess its performance.

The plot above shows two lines: one for the actual values from the test set (in blue) and one for the predicted values generated by the RNN model (in orange, dashed). By comparing these lines, you can visually assess how closely the model's predictions follow the true values over time. Ideally, the predicted line should closely track the actual line, indicating that the model is accurately capturing the underlying patterns in the time series. Significant deviations between the two lines may suggest areas where the model could be improved, such as by tuning hyperparameters, increasing the model complexity, or providing more training data. This kind of visualization is a valuable diagnostic tool for understanding model performance beyond just numerical metrics like RMSE.

Building a Basic RNN Model with PyTorch

Introduction to Building a Basic RNN Model

Defining the RNN Model

Splitting the Data

Join the 1M+ learners on CodeSignal

Training the RNN Model

Evaluating the RNN Model

Making Predictions and Visualizing Results

Summary and Preparation for Practice