Introduction to Time Series Forecasting with GRUs using PyTorch

Introduction to Time Series Forecasting with GRUs

Welcome to the first lesson of the course on Time Series Forecasting with GRUs . In this lesson, we will explore the concept of time series forecasting, which involves predicting future values based on previously observed data points. This is particularly useful in fields like finance, weather prediction, and air quality monitoring. We will focus on using Gated Recurrent Units (GRUs), a type of recurrent neural network, to handle univariate time series data. GRUs are well-suited for this task because they can capture temporal dependencies in data efficiently. To make our learning concrete, we will use the Airline Passenger Traffic dataset . This dataset contains monthly totals of international airline passengers from 1949 to 1960, and we will use it to predict future passenger numbers. By the end of this lesson, you will understand how to preprocess data, build a GRU model, and train it for forecasting.

Data Processing Recap

Before we dive into building our GRU model, let's quickly recap the data processing steps. We will use the Airline Passenger Traffic dataset, which is a well-known example of time series data. This dataset is valuable for illustrating trends, seasonality, and other time series characteristics. We have stored the dataset in a CSV file named data.csv, which includes a 'Month' column representing the time intervals and a 'Passengers' column representing the number of passengers. You can load and preprocess the dataset using the following code: Python import numpy as np import pandas as pd from sklearn.preprocessing import MinMaxScaler def preprocess_data(filepath, seq_length=10): """Preprocess time series data for RNN training.""" data = pd.read_csv(filepath) data['Month'] = pd.to_datetime(data['Month']) data.set_index('Month', inplace=True) scaler = MinMaxScaler() normalized_data = scaler.fit_transform(data) X, y = [], [] for i in range(len(normalized_data) - seq_length): X.append(normalized_data[i:i+seq_length]) y.append(normalized_data[i+seq_length]) X, y = np.array(X), np.array(y) # Reshape X to ensure it has the correct shape for LSTM input X = X.reshape((X.shape[0], X.shape[1], 1)) return X, y, scaler seq_length = 10 X, y, scaler = preprocess_data('data.csv', seq_length) In this code, we load the dataset, convert the 'Month' column to a datetime format, and set it as the index. We then normalize the data using MinMaxScaler, which scales the data to a range between 0 and 1. This preprocessing is crucial for ensuring that our model can learn effectively from the data.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Building a GRU Model for Univariate Forecasting

Now that we have our data and understand how GRUs work, let's build a GRU model using PyTorch. GRUs are a type of recurrent neural network that can capture temporal dependencies in sequential data. Here's how you can define a GRU model:

Python

import torch
import torch.nn as nn

class GRUModel(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(GRUModel, self).__init__()
        self.gru1 = nn.GRU(input_size, hidden_size, batch_first=True)
        self.gru2 = nn.GRU(hidden_size, hidden_size // 2, batch_first=True)
        self.fc = nn.Linear(hidden_size // 2, output_size)

    def forward(self, x):
        x, _ = self.gru1(x)
        x, _ = self.gru2(x)
        x = x[:, -1, :]  # Take the last time step
        x = self.fc(x)
        return x

# Define model parameters
input_size = 1  # Univariate input
hidden_size = 32
output_size = 1

# Instantiate the model
model = GRUModel(input_size, hidden_size, output_size)

import torch
import torch.nn as nn

class GRUModel(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(GRUModel, self).__init__()
        self.gru1 = nn.GRU(input_size, hidden_size, batch_first=True)
        self.gru2 = nn.GRU(hidden_size, hidden_size // 2, batch_first=True)
        self.fc = nn.Linear(hidden_size // 2, output_size)

    def forward(self, x):
        x, _ = self.gru1(x)
        x, _ = self.gru2(x)
        x = x[:, -1, :]  # Take the last time step
        x = self.fc(x)
        return x

# Define model parameters
input_size = 1  # Univariate input
hidden_size = 32
output_size = 1

# Instantiate the model
model = GRUModel(input_size, hidden_size, output_size)

In this model, we use two GRU layers. The first GRU layer has 32 units and returns sequences, which means it outputs a sequence of data points for each input sequence. The second GRU layer has 16 units and does not return sequences, as it outputs a single data point. Finally, we use a Linear layer with one unit to predict the number of passengers. Our target is the number of passengers, which we aim to forecast.

Training the GRU Model

With our model defined, we can now compile and train it. Compiling the model involves specifying the optimizer and loss function. We will use the Adam optimizer and mean squared error (MSE) as the loss function. Here's how you can train the model:

Python

import torch.optim as optim

# Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

# Convert data to PyTorch tensors
X_tensor = torch.tensor(X, dtype=torch.float32)
y_tensor = torch.tensor(y, dtype=torch.float32)

# Training loop
epochs = 10
batch_size = 16

for epoch in range(epochs):
    model.train()
    for i in range(0, len(X_tensor), batch_size):
        X_batch = X_tensor[i:i+batch_size]
        y_batch = y_tensor[i:i+batch_size]

        # Forward pass
        outputs = model(X_batch)
        loss = criterion(outputs.squeeze(), y_batch.squeeze())  # Ensure both are of shape [batch_size]

        # Backward pass and optimization
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

    print(f'Epoch [{epoch+1}/{epochs}], Loss: {loss.item():.4f}')

import torch.optim as optim

# Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

# Convert data to PyTorch tensors
X_tensor = torch.tensor(X, dtype=torch.float32)
y_tensor = torch.tensor(y, dtype=torch.float32)

# Training loop
epochs = 10
batch_size = 16

for epoch in range(epochs):
    model.train()
    for i in range(0, len(X_tensor), batch_size):
        X_batch = X_tensor[i:i+batch_size]
        y_batch = y_tensor[i:i+batch_size]

        # Forward pass
        outputs = model(X_batch)
        loss = criterion(outputs.squeeze(), y_batch.squeeze())  # Ensure both are of shape [batch_size]

        # Backward pass and optimization
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

    print(f'Epoch [{epoch+1}/{epochs}], Loss: {loss.item():.4f}')

In this code, we define the loss function and optimizer:

Loss Function: We use mean squared error (MSE) as the loss function, which measures how well the model's predictions match the actual data. It quantifies the difference between predicted and true values, guiding the optimizer in adjusting the model to improve accuracy. MSE is commonly used for regression tasks as it penalizes larger errors more heavily.
Optimizer: We use the Adam optimizer, which is an algorithm that adjusts the model's weights to minimize the loss function. It determines how the model learns from the data by updating weights during training. Adam is popular due to its efficiency and ability to handle sparse gradients.

We then train the model using a training loop, specifying the number of epochs (the number of times the model will iterate over the entire dataset) and the batch size (the number of samples processed before the model is updated). Training the model allows it to learn patterns in the data and make accurate predictions.

Feature	Basic RNNs	LSTMs	GRUs
Architecture	Simple recurrent structure	Memory cells with input, output, and forget gates	Simplified version of LSTM with update and reset gates
Handling Long-Term Dependencies	Limited due to vanishing gradient problem	Effective with memory cells and gates	Effective with fewer gates and simpler structure
Complexity	Low	High due to multiple gates	Moderate, fewer gates than LSTM
Training Time	Fast	Slower due to complexity	Faster than LSTM, slower than basic RNN
Performance	Limited for long sequences	High for capturing long-term dependencies	Comparable to LSTM, often preferred for efficiency
Use Cases	Short sequences, simple tasks	Tasks requiring long-term context	Efficient for a wide range of tasks, especially with limited resources