Evaluating and Visualizing LSTM Model Performance | CodeSignal Learn

Evaluating and Visualizing LSTM Model Performance | CodeSignal Learn

Introduction to Evaluating LSTM Models

Welcome to the next step in your journey through the "Time Series Forecasting with LSTMs" course. In this lesson, we will focus on evaluating LSTM models to understand their performance in time series forecasting tasks. As you may recall from previous lessons, LSTMs are powerful tools for capturing temporal dependencies in sequence data. However, to ensure that our models are effective, we need to evaluate their predictions accurately. This lesson will guide you through the process of assessing prediction accuracy using evaluation metrics and visualizing the results to gain insights into model performance.

Understanding RMSE as an Evaluation Metric

\text{RMSE} = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2}

Understanding MAE as an Evaluation Metric

\text{MAE} = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i|

Understanding MAPE as an Evaluation Metric

\text{MAPE} = \frac{100}{n} \sum_{i=1}^{n} \left| \frac{y_i - \hat{y}_i}{y_i} \right|

Understanding R² Score as an Evaluation Metric

R^2 = 1 - \frac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2}

Example: Predicting and Evaluating with LSTM

Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Let's walk through an example of predicting temperature using an LSTM model and evaluating its performance using RMSE, MAE, MAPE, and R² Score. Suppose we have already trained an LSTM model on temperature data. We can use this model to make predictions and then compute these metrics to evaluate its accuracy. import numpy as np from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score # Predict y_pred = model.predict(X) # Compute RMSE rmse = np.sqrt(mean_squared_error(y_actual_rescaled, y_pred_rescaled)) print(f"RMSE: {rmse}") # Compute MAE mae = mean_absolute_error(y_actual_rescaled, y_pred_rescaled) print(f"MAE: {mae}") # Compute MAPE mape = np.mean(np.abs((y_actual_rescaled - y_pred_rescaled) / y_actual_rescaled)) * 100 print(f"MAPE: {mape}%") # Compute R² Score r2 = r2_score(y_actual_rescaled, y_pred_rescaled) print(f"R² Score: {r2}") In this example, we first use the predict method of the trained LSTM model to generate predictions for the input data X. We then calculate the RMSE, MAE, MAPE, and R² Score using functions from the sklearn library and basic numpy operations. These metrics provide a comprehensive evaluation of the model's prediction accuracy, allowing you to assess its performance from different perspectives.

Visualizing LSTM Predictions

Visualizing the predictions of an LSTM model is an essential step in understanding its performance. By plotting the actual and predicted values, you can gain insights into how well the model captures the underlying patterns in the data. Visualization helps identify trends, discrepancies, and areas where the model may need improvement. Let's see how to create a plot to visualize the predictions. import matplotlib.pyplot as plt # Plot predictions plt.figure(figsize=(10, 5)) plt.plot(y_actual_rescaled, label="Actual Temperature") plt.plot(y_pred_rescaled, label="Predicted Temperature", linestyle='dashed') plt.title("LSTM Predictions vs. Actual") plt.legend() plt.show() In this example, we use the matplotlib library to create a plot of the actual and predicted temperature values. The plot function is used to draw lines representing the actual and predicted values, with different line styles for clarity. The legend function adds a legend to the plot, making it easy to distinguish between the actual and predicted values. By using different line styles and labels, the plot becomes more readable and informative, allowing you to easily compare the model's predictions with the actual values. By visualizing the predictions, you can assess the model's performance and identify any discrepancies between the predicted and actual values.

Python

import numpy as np
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score

# Predict
y_pred = model.predict(X)

# Compute RMSE
rmse = np.sqrt(mean_squared_error(y_actual_rescaled, y_pred_rescaled))
print(f"RMSE: {rmse}")

# Compute MAE
mae = mean_absolute_error(y_actual_rescaled, y_pred_rescaled)
print(f"MAE: {mae}")

# Compute MAPE
mape = np.mean(np.abs((y_actual_rescaled - y_pred_rescaled) / y_actual_rescaled)) * 100
print(f"MAPE: {mape}%")

# Compute R² Score
r2 = r2_score(y_actual_rescaled, y_pred_rescaled)
print(f"R² Score: {r2}")

Python

import matplotlib.pyplot as plt

# Plot predictions
plt.figure(figsize=(10, 5))
plt.plot(y_actual_rescaled, label="Actual Temperature")
plt.plot(y_pred_rescaled, label="Predicted Temperature", linestyle='dashed')
plt.title("LSTM Predictions vs. Actual")
plt.legend()
plt.show()