Handling Multivariate Time Series with RNNs Using PyTorch

Introduction to Multivariate Time Series

Welcome to the first lesson of the course "Handling Multivariate Time Series with RNNs." In this lesson, we will lay the groundwork for understanding and preprocessing multivariate time series data. A multivariate time series consists of multiple variables recorded over time, which is crucial for forecasting as it considers interactions between different variables for more accurate predictions.

Introduction to the Air Quality Dataset

We will be working with the Air Quality dataset from the UCI Machine Learning Repository, accessible here . This dataset contains 9,358 instances of hourly averaged responses from a gas multisensor device deployed in an Italian city. It includes 15 features, such as concentrations of air pollutants (e.g., CO, NOx, C6H6) and meteorological data (e.g., temperature, humidity). The data was recorded from March 2004 to February 2005. The dataset uses -200 to represent missing values, which we will replace with NaN during preprocessing to ensure accurate data handling. Our goal is to perform multivariate time series analysis, which involves handling missing values to ensure accurate modeling and predictions.

Loading the Dataset

To begin, we need to load the Air Quality dataset into our environment. We will use the pandas library, which is a powerful tool for data manipulation and analysis in Python. The dataset is hosted on an AWS S3 URL, and we can load it directly from there. It's important to handle missing values during the loading process to ensure the integrity of our data. Here's how you can load the dataset: Python import pandas as pd import numpy as np # Load dataset from AWS S3 URL url = "https://codesignal-staging-assets.s3.amazonaws.com/uploads/1742293523899/AirQualityUCI.csv" df = pd.read_csv(url, sep=';', decimal=',') # Replace -200 with NaN df.replace(-200, np.nan, inplace=True) In this code, we use pd.read_csv() to read the CSV file from the specified URL. The sep=';' parameter indicates that the data is separated by semicolons, and decimal=',' specifies that commas are used as decimal points. We also replace -200 with NaN to correctly identify missing values. This step is crucial for correctly interpreting the numerical data.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Cleaning the Dataset

Cleaning the dataset involves removing unnecessary columns and handling missing values. This step is vital to ensure that our data is ready for analysis and modeling. Here are the main points to consider:

Dropping the original Date and Time columns:
After combining these into a single DateTime column, the original columns are no longer needed and can be removed to avoid redundancy.

Python

# Drop the original 'Date' and 'Time' columns
df.drop(columns=['Date', 'Time'], inplace=True)

# Drop the original 'Date' and 'Time' columns
df.drop(columns=['Date', 'Time'], inplace=True)

Removing unnecessary columns:
The columns Unnamed: 15 and Unnamed: 16 are dropped because they do not contain any useful information. These columns are artifacts from the CSV file format and are either empty or filled entirely with missing values.
Python
```
# Drop unnecessary columns
df.drop(columns=['Unnamed: 15', 'Unnamed: 16'], inplace=True)
```
```
# Drop unnecessary columns
df.drop(columns=['Unnamed: 15', 'Unnamed: 16'], inplace=True)
```
Dropping rows with missing values in essential features:
For key features such as CO(GT), NO2(GT), T, and RH, we drop any rows where these values are missing. These features are critical for our analysis and modeling, and missing values in these columns would make it difficult to interpret or accurately impute the data. Removing these rows ensures that our core variables are complete and reliable.
Python
```
# Drop rows where essential features are missing
df.dropna(subset=['CO(GT)', 'NO2(GT)', 'T', 'RH'], inplace=True)
```
```
# Drop rows where essential features are missing
df.dropna(subset=['CO(GT)', 'NO2(GT)', 'T', 'RH'], inplace=True)
```
Filling remaining missing values:
For other features, we use forward-fill and backward-fill techniques to handle missing values. Forward-fill propagates the last valid observation forward, which is useful when the previous value is a reasonable estimate for the missing data. Backward-fill is then used to fill any remaining gaps by propagating the next valid observation backward. This two-step approach helps maintain the continuity and integrity of the time series data.
Python
```
# Fill missing values using forward-fill & backward-fill
df.ffill(inplace=True)
df.bfill(inplace=True)
```
```
# Fill missing values using forward-fill & backward-fill
df.ffill(inplace=True)
df.bfill(inplace=True)
```

These steps ensure that our dataset is clean and complete, ready for further analysis.

Introduction to Multivariate Time Series

Introduction to the Air Quality Dataset

Loading the Dataset

Join the 1M+ learners on CodeSignal

Combining Date and Time Columns

Cleaning the Dataset

Setting the DateTime Index

Visualizing the Data

Displaying the Cleaned Dataset

Summary and Next Steps