Building & Training GRUs for Multivariate Forecasting | CodeSignal Learn

Building & Training GRUs for Multivariate Forecasting | CodeSignal Learn

Introduction to Time Series Forecasting with GRUs

Welcome to the first lesson of the course on Time Series Forecasting with GRUs . In this lesson, we will explore the concept of time series forecasting, which involves predicting future values based on previously observed data points. This is particularly useful in fields like finance, weather prediction, and air quality monitoring. We will focus on using Gated Recurrent Units (GRUs), a type of recurrent neural network, to handle multivariate time series data. GRUs are well-suited for this task because they can capture temporal dependencies in data efficiently. To make our learning concrete, we will use the UCI Air Quality dataset . This dataset contains various air quality measurements, and we will use it to predict future temperature values. By the end of this lesson, you will understand how to preprocess data, build a GRU model, and train it for forecasting.

Data Processing Recap

Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Building a GRU Model for Multivariate Forecasting

Now that we have our data and understand how GRUs work, let's build a GRU model using TensorFlow and Keras . GRUs are a type of recurrent neural network that can capture temporal dependencies in sequential data. Here's how you can define a GRU model: import tensorflow as tf from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Input, GRU, Dense # Define a GRU model for multivariate forecasting model = Sequential([Input(shape=(sequence_length, len(features))), GRU(32, activation="tanh", return_sequences=True), GRU(16, activation="tanh"), Dense(1)]) In this model, we use two GRU layers. The first GRU layer has 32 units and returns sequences, which means it outputs a sequence of data points for each input sequence. The second GRU layer has 16 units and does not return sequences, as it outputs a single data point. Finally, we use a Dense layer with one unit to predict the temperature. Our features include 'CO(GT)', 'NO2(GT)', 'PT08.S5(O3)', 'RH', and 'T', and our target is the temperature ('T'), which we aim to forecast.

Training the GRU Model

With our model defined, we can now compile and train it. Compiling the model involves specifying the optimizer and loss function. We will use the Adam optimizer and mean squared error (MSE) as the loss function. Here's how you can train the model: model.compile(optimizer="adam", loss="mse") # Train model model.fit(X, y, epochs=10, batch_size=16) In this code, we compile the model by specifying the optimizer and loss function : Optimizer : We use the Adam optimizer, which is an algorithm that adjusts the model's weights to minimize the loss function. It determines how the model learns from the data by updating weights during training. Adam is popular due to its efficiency and ability to handle sparse gradients. Loss Function : We use mean squared error (MSE) as the loss function, which measures how well the model's predictions match the actual data. It quantifies the difference between predicted and true values, guiding the optimizer in adjusting the model to improve accuracy. MSE is commonly used for regression tasks as it penalizes larger errors more heavily. We then train the model using the fit method, specifying the number of epochs (the number of times the model will iterate over the entire dataset) and the batch size (the number of samples processed before the model is updated). Training the model allows it to learn patterns in the data and make accurate predictions.

Comparing RNN, LSTM, and GRU

To better understand the differences between basic RNNs, LSTMs, and GRUs, let's compare their key features in the table below: Feature Basic RNNs LSTMs GRUs Architecture Simple recurrent structure Memory cells with input, output, and forget gates Simplified version of LSTM with update and reset gates Handling Long-Term Dependencies Limited due to vanishing gradient problem Effective with memory cells and gates Effective with fewer gates and simpler structure Complexity Low High due to multiple gates Moderate, fewer gates than LSTM Training Time Fast Slower due to complexity Faster than LSTM, slower than basic RNN Performance Limited for long sequences High for capturing long-term dependencies Comparable to LSTM, often preferred for efficiency Use Cases Short sequences, simple tasks Tasks requiring long-term context Efficient for a wide range of tasks, especially with limited resources In summary, while basic RNNs are suitable for simple tasks with short sequences, LSTMs and GRUs are better equipped to handle long-term dependencies. GRUs offer a good balance between complexity and performance, making them a popular choice for many time series forecasting applications.

Python

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Input, GRU, Dense

# Define a GRU model for multivariate forecasting
model = Sequential([
    Input(shape=(sequence_length, len(features))),
    GRU(32, activation="tanh", return_sequences=True),
    GRU(16, activation="tanh"),
    Dense(1)
])