Extending RNNs for Time Series Classification Tasks

Introduction

In this lesson, we will explore how to extend Recurrent Neural Networks (RNNs) for time series classification tasks. Time series classification involves predicting categorical labels based on sequential data. We will use a dataset containing monthly airline passenger numbers to demonstrate the process of loading and preparing data, building an RNN classification model, and evaluating its performance. By the end of this lesson, you will have a solid understanding of how to apply RNNs to classify time series data.

Loading and Preparing Data for Classification

To begin, we need to load our time series data and prepare it for classification tasks. We'll use a dataset containing monthly airline passenger numbers as an example. The first step is to load the data and preprocess it to create input sequences and corresponding labels. Python import pandas as pd import numpy as np from sklearn.preprocessing import MinMaxScaler # Load dataset df = pd.read_csv('AirPassengers.csv') # Ensure the column name matches df.columns = ['Month', 'Passengers'] # Generate labels BEFORE scaling df['Label'] = (df['Passengers'].diff().shift(-1) > 0).astype(int) # Create binary labels: 1 if next month's passengers increase, else 0 # Normalize the data scaler = MinMaxScaler(feature_range=(0, 1)) df['Passengers'] = scaler.fit_transform(df['Passengers'].values.reshape(-1, 1)) # Create input sequences def create_sequences(data, labels, seq_length): X, y = [], [] for i in range(len(data) - seq_length): X.append(data[i:i + seq_length]) y.append(labels[i + seq_length]) # Correct alignment return np.array(X), np.array(y) seq_length = 10 X, y = create_sequences(df['Passengers'].values, df['Label'].values, seq_length) In this code, we load the dataset using pandas and ensure the column names match. We generate binary labels indicating whether the next value in the time series is higher or lower before scaling the data. We then normalize the passenger numbers to a range between 0 and 1 using MinMaxScaler . The function create_sequences generates input sequences of a specified length (seq_length) and their corresponding labels, returning the input sequences X and the target values y .

Data Preparation for Classification

Next, we prepare the data specifically for classification by converting the labels to a categorical format. Python from tensorflow.keras.utils import to_categorical # Convert labels to categorical format y_classification = to_categorical(y) Here, we convert the binary labels to a categorical format using to_categorical, which is necessary for training the classification model.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Building the RNN Classification Model

With our data prepared, we can now define the RNN model for classification.

Python

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import SimpleRNN, Dense, Dropout, Input
from tensorflow.keras.optimizers import Adam

# Define improved RNN model
model_classification = Sequential([
    Input(shape=(seq_length, 1)),
    SimpleRNN(20, activation='relu', return_sequences=True),  # More units for better learning
    Dropout(0.2),  # Helps prevent overfitting
    SimpleRNN(10, activation='relu'),
    Dense(2, activation='softmax')  # 2 classes: Up or Down
])

# Compile the model
model_classification.compile(optimizer=Adam(learning_rate=0.001),
                             loss='categorical_crossentropy',
                             metrics=['accuracy'])

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import SimpleRNN, Dense, Dropout, Input
from tensorflow.keras.optimizers import Adam

# Define improved RNN model
model_classification = Sequential([
    Input(shape=(seq_length, 1)),
    SimpleRNN(20, activation='relu', return_sequences=True),  # More units for better learning
    Dropout(0.2),  # Helps prevent overfitting
    SimpleRNN(10, activation='relu'),
    Dense(2, activation='softmax')  # 2 classes: Up or Down
])

# Compile the model
model_classification.compile(optimizer=Adam(learning_rate=0.001),
                             loss='categorical_crossentropy',
                             metrics=['accuracy'])

We define a Sequential model with an Input layer specifying the shape of the input data, followed by a SimpleRNN layer with 20 units and a ReLU activation function, and a Dropout layer to help prevent overfitting. Another SimpleRNN layer with 10 units is added, followed by a Dense layer with 2 units and a softmax activation function. The softmax function is used to convert the output into a probability distribution over the two classes (up or down), ensuring that the sum of the probabilities is 1. We compile the model using the Adam optimizer and categorical_crossentropy loss function. The categorical_crossentropy loss function is used for multi-class classification problems. It measures the dissimilarity between the true label distribution and the predicted probability distribution. By minimizing this loss, the model learns to assign higher probabilities to the correct classes, improving its classification accuracy.

Training and Evaluating the Classification Model

Finally, we train and evaluate the model using the prepared data.

Python

from tensorflow.keras.callbacks import ReduceLROnPlateau

# Reduce learning rate if no improvement
reduce_lr = ReduceLROnPlateau(monitor='loss', factor=0.5, patience=5, min_lr=0.0001)

# Train the model
model_classification.fit(X, y_classification, epochs=5, batch_size=32, verbose=1, callbacks=[reduce_lr])

# Predict class probabilities
y_pred_class = model_classification.predict(X)
y_pred_labels = np.argmax(y_pred_class, axis=1)

# Compute accuracy correctly
accuracy = np.mean(y_pred_labels == y)
print("Final Model Accuracy: {:.2f}%".format(accuracy * 100))

from tensorflow.keras.callbacks import ReduceLROnPlateau

# Reduce learning rate if no improvement
reduce_lr = ReduceLROnPlateau(monitor='loss', factor=0.5, patience=5, min_lr=0.0001)

# Train the model
model_classification.fit(X, y_classification, epochs=5, batch_size=32, verbose=1, callbacks=[reduce_lr])

# Predict class probabilities
y_pred_class = model_classification.predict(X)
y_pred_labels = np.argmax(y_pred_class, axis=1)

# Compute accuracy correctly
accuracy = np.mean(y_pred_labels == y)
print("Final Model Accuracy: {:.2f}%".format(accuracy * 100))

In this section, we train and evaluate the RNN classification model. We use the fit method to train the model with the input sequences X and the categorical labels y_classification. A ReduceLROnPlateau callback is included to reduce the learning rate if no improvement in the loss is observed, which can help in achieving better convergence.

After training, we use the predict method to obtain class probabilities for the input sequences. The argmax function is then applied to these probabilities to determine the predicted class labels (y_pred_labels), selecting the class with the highest probability for each sequence.

We compute the model's accuracy by comparing the predicted labels with the actual labels (y). Accuracy is a key metric that represents the proportion of correct predictions made by the model. It is calculated as the number of correct predictions divided by the total number of predictions. A higher accuracy indicates better model performance in classifying the time series data. Finally, we print the final model accuracy as a percentage to provide a clear understanding of the model's effectiveness.

Extending RNNs for Time Series Classification Tasks

Introduction

Loading and Preparing Data for Classification

Data Preparation for Classification

Join the 1M+ learners on CodeSignal

Building the RNN Classification Model

Training and Evaluating the Classification Model

Summary