Evaluating and Visualizing LSTM Model Performance

Introduction to Evaluating LSTM Models

Welcome to the next step in your journey through the "Time Series Forecasting with LSTMs" course. In this lesson, we will focus on evaluating LSTM models to understand their performance in time series forecasting tasks. As you may recall from previous lessons, LSTMs are powerful tools for capturing temporal dependencies in sequence data. However, to ensure that our models are effective, we need to evaluate their predictions accurately. This lesson will guide you through the process of assessing prediction accuracy using evaluation metrics and visualizing the results to gain insights into model performance.

Understanding RMSE as an Evaluation Metric

\text{RMSE} = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2}

Understanding MAE as an Evaluation Metric

\text{MAE} = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i|

Understanding MAPE as an Evaluation Metric

\text{MAPE} = \frac{100}{n} \sum_{i=1}^{n} \left| \frac{y_i - \hat{y}_i}{y_i} \right|

Understanding R² Score as an Evaluation Metric

R^2 = 1 - \frac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2}

Example: Predicting and Evaluating with LSTM

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Python

import numpy as np
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score

# Predict
y_pred = model.predict(X)

# Compute RMSE
rmse = np.sqrt(mean_squared_error(y_actual_rescaled, y_pred_rescaled))
print(f"RMSE: {rmse}")

# Compute MAE
mae = mean_absolute_error(y_actual_rescaled, y_pred_rescaled)
print(f"MAE: {mae}")

# Compute MAPE
mape = np.mean(np.abs((y_actual_rescaled - y_pred_rescaled) / y_actual_rescaled)) * 100
print(f"MAPE: {mape}%")

# Compute R² Score
r2 = r2_score(y_actual_rescaled, y_pred_rescaled)
print(f"R² Score: {r2}")

import numpy as np
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score

# Predict
y_pred = model.predict(X)

# Compute RMSE
rmse = np.sqrt(mean_squared_error(y_actual_rescaled, y_pred_rescaled))
print(f"RMSE: {rmse}")

# Compute MAE
mae = mean_absolute_error(y_actual_rescaled, y_pred_rescaled)
print(f"MAE: {mae}")

# Compute MAPE
mape = np.mean(np.abs((y_actual_rescaled - y_pred_rescaled) / y_actual_rescaled)) * 100
print(f"MAPE: {mape}%")

# Compute R² Score
r2 = r2_score(y_actual_rescaled, y_pred_rescaled)
print(f"R² Score: {r2}")

Python

import matplotlib.pyplot as plt

# Plot predictions
plt.figure(figsize=(10, 5))
plt.plot(y_actual_rescaled, label="Actual Temperature")
plt.plot(y_pred_rescaled, label="Predicted Temperature", linestyle='dashed')
plt.title("LSTM Predictions vs. Actual")
plt.legend()
plt.show()

import matplotlib.pyplot as plt

# Plot predictions
plt.figure(figsize=(10, 5))
plt.plot(y_actual_rescaled, label="Actual Temperature")
plt.plot(y_pred_rescaled, label="Predicted Temperature", linestyle='dashed')
plt.title("LSTM Predictions vs. Actual")
plt.legend()
plt.show()

Python

import numpy as np
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score

# Predict
y_pred = model.predict(X)

# Compute RMSE
rmse = np.sqrt(mean_squared_error(y_actual_rescaled, y_pred_rescaled))
print(f"RMSE: {rmse}")

# Compute MAE
mae = mean_absolute_error(y_actual_rescaled, y_pred_rescaled)
print(f"MAE: {mae}")

# Compute MAPE
mape = np.mean(np.abs((y_actual_rescaled - y_pred_rescaled) / y_actual_rescaled)) * 100
print(f"MAPE: {mape}%")

# Compute R² Score
r2 = r2_score(y_actual_rescaled, y_pred_rescaled)
print(f"R² Score: {r2}")

Python

import matplotlib.pyplot as plt

# Plot predictions
plt.figure(figsize=(10, 5))
plt.plot(y_actual_rescaled, label="Actual Temperature")
plt.plot(y_pred_rescaled, label="Predicted Temperature", linestyle='dashed')
plt.title("LSTM Predictions vs. Actual")
plt.legend()
plt.show()

Introduction to Evaluating LSTM Models

Understanding RMSE as an Evaluation Metric

Understanding MAE as an Evaluation Metric

Understanding MAPE as an Evaluation Metric

Understanding R² Score as an Evaluation Metric

Example: Predicting and Evaluating with LSTM

Join the 1M+ learners on CodeSignal

Visualizing LSTM Predictions

Summary and Preparation for Practice