Building a Basic RNN Model with PyTorch

Introduction to Building a Basic RNN Model

Welcome to the next step in your journey of mastering Recurrent Neural Networks (RNNs) for time series analysis. In the previous lesson, you learned how to prepare time series data for RNNs by normalizing it and converting it into sequences. This foundation is crucial as we now move on to building and evaluating a basic RNN model. In this lesson, you will learn how to define, train, and evaluate a simple RNN model using PyTorch . By the end of this lesson, you will be able to implement a basic RNN model to predict time series values and assess its performance.

Defining the RNN Model

Let's start by defining a basic RNN model. We will use PyTorch, which is a powerful tool for building neural networks. The model will consist of an RNN layer, followed by a Linear layer. The RNN layer is responsible for processing the sequences of data, while the Linear layer outputs the prediction. Here's how you can define the model: Python import torch import torch.nn as nn class SimpleRNNModel(nn.Module): def __init__(self, input_size, hidden_size, output_size): super(SimpleRNNModel, self).__init__() self.rnn = nn.RNN(input_size, hidden_size, batch_first=True) self.linear = nn.Linear(hidden_size, output_size) def forward(self, x): out, _ = self.rnn(x) out = self.linear(out[:, -1, :]) # Take the last output of the sequence return out # Define RNN model input_size = 1 hidden_size = 10 output_size = 1 model = SimpleRNNModel(input_size, hidden_size, output_size) In this code, we define a class SimpleRNNModel that subclasses nn.Module . The model consists of an RNN layer with 10 hidden units and a Linear layer to produce the final output. The forward method defines the forward pass of the model, where we take the last output of the RNN sequence to pass through the Linear layer.

Splitting the Data

For time series data, it's important to split the data chronologically to avoid data leakage. This means using the earlier part of the series for training and the later part for testing. Here's how you can split the data: Python # Determine split index split_idx = int(len(X) * 0.8) # Chronological split X_train, X_test = X[:split_idx], X[split_idx:] y_train, y_test = y[:split_idx], y[split_idx:] In this code, we use the first 80% of the data for training and the remaining 20% for testing, preserving the temporal order of the time series. This approach ensures that the model is always tested on data points that occur after those it was trained on, which is essential for time series forecasting.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Training the RNN Model

With the model defined and data split, the next step is to train it using the training dataset. Training involves feeding the model with input sequences and adjusting its parameters to minimize the loss function.

Here's how you can train the model:

Python

import torch.optim as optim

# Define loss function and optimizer
criterion = nn.MSELoss()  # Mean Squared Error loss for regression tasks
optimizer = optim.Adam(model.parameters(), lr=0.001)  # Adam optimizer for updating model weights

# Training loop parameters
epochs = 25
batch_size = 16

# List to store loss values for each epoch (for plotting later)
losses = []

for epoch in range(epochs):
    model.train()  # Set the model to training mode
    epoch_loss = 0  # Track loss for this epoch

    # Loop over the training data in batches
    for i in range(0, len(X_train), batch_size):
        # Prepare batch data as PyTorch tensors
        X_batch = torch.tensor(X_train[i:i+batch_size], dtype=torch.float32)
        y_batch = torch.tensor(y_train[i:i+batch_size], dtype=torch.float32)

        # ---- Forward Pass ----
        # Pass the input batch through the model to get predictions
        outputs = model(X_batch)

        # Compute the loss between predictions and actual values
        loss = criterion(outputs, y_batch)

        # ---- Backpropagation ----
        # Zero the gradients from the previous step
        optimizer.zero_grad()
        # Compute gradients of the loss with respect to model parameters
        loss.backward()
        # Update model parameters using the optimizer
        optimizer.step()

        # Accumulate loss for this batch
        epoch_loss += loss.item()

    # Calculate average loss for the epoch and store it
    avg_loss = epoch_loss / (len(X_train) // batch_size)
    losses.append(avg_loss)
    print(f'Epoch [{epoch+1}/{epochs}], Loss: {avg_loss:.4f}')

Explanation:

Forward Pass:
In the forward pass, the input batch (X_batch) is passed through the model (model(X_batch)). The model processes the input data and produces predictions (outputs). These predictions are then compared to the actual target values (y_batch) using the loss function (criterion). The result is a single loss value that quantifies how well the model's predictions match the actual values.
Backpropagation:
Backpropagation is the process of updating the model's parameters to minimize the loss. First, we clear any previously stored gradients with optimizer.zero_grad(). Then, loss.backward() computes the gradients of the loss with respect to each model parameter. Finally, optimizer.step() updates the model parameters using these gradients, moving them in the direction that reduces the loss.
Batch Training:
The training data is processed in small batches (here, of size 16) rather than all at once. This helps the model learn more efficiently and makes training feasible for large datasets.
Epochs:
The entire training dataset is processed multiple times (here, 25 epochs) to allow the model to learn from the data.
Loss Tracking:
The average loss for each epoch is stored in the losses list, which can be used later to plot the training loss curve and monitor the model's learning progress.

Introduction to Building a Basic RNN Model

Defining the RNN Model

Splitting the Data

Join the 1M+ learners on CodeSignal

Training the RNN Model

Evaluating the RNN Model

Making Predictions and Visualizing Results

Summary and Preparation for Practice