Neural Network Forward Propagation

Introduction to Neural Network Operations

Hello, and welcome to today's exciting lesson! We will delve into the world of neural networks, focusing on a technique called forward propagation, or the data flow from input to output in a neural network. Neural networks are a variety of machine learning models inspired by the human brain. They draw upon the idea of having neurons interconnected in a net-like structure to process and learn from information, similar to how our brain learns from the data fed into it by our senses. One basic and essential step in how a neural network processes and learns from information is termed forward propagation . As the name suggests, forward propagation involves moving forward through the network. Each node in the network receives inputs from the nodes in the previous layer, multiplies them by their weights, adds a bias, and then "fires" that through an activation function. The result is then passed on as input to the nodes in the next layer. This process is repeated layer after layer until we reach the output layer, giving us the predicted output. But what if the predicted output is far from the actual result? That's when backpropagation comes into play. In simple terms, backpropagation is the method used to update the weights of our neural network based on error correction. The less the error, the better our model's predictions. The entity that quantifies the error between predicted and actual outputs is the loss function . To minimize this loss and hence the prediction error, we use optimization algorithms like gradient descent . In this lesson, we focus on understanding forward propagation, setting a solid foundation for learning more intricate neural network operations such as backpropagation in future lessons.

Working with the Iris Dataset

Now, let's get our hands on practical implementation. We'll use the built-in iris dataset in R for our demonstration: R # Load required libraries library(keras3) library(dplyr) library(ggplot2) set.seed(42) # Load the iris dataset data(iris) # Only use the first two features and first two classes iris_binary <- iris %>% filter(Species %in% c("setosa", "versicolor")) %>% mutate(Species = as.numeric(Species == "versicolor")) # 0 for setosa, 1 for versicolor X <- as.matrix(iris_binary[, 1:2]) # Sepal.Length and Sepal.Width y <- iris_binary$Species We start by loading the iris dataset, which is a classic dataset in statistics and machine learning. For this task, we only consider the first two features: sepal length and sepal width. We also convert the problem into a binary classification problem by selecting only the "setosa" and "versicolor" classes.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

# Scale the inputs to (0, 1) range
X_scaled <- scale(X, center = TRUE, scale = TRUE)
X_scaled <- as.matrix(X_scaled)

# Split into train and test sets (80/20 split)
set.seed(42)
train_indices <- sample(1:nrow(X_scaled), size = 0.8 * nrow(X_scaled))
X_train <- X_scaled[train_indices, ]
X_test <- X_scaled[-train_indices, ]
y_train <- y[train_indices]
y_test <- y[-train_indices]

# Scale the inputs to (0, 1) range
X_scaled <- scale(X, center = TRUE, scale = TRUE)
X_scaled <- as.matrix(X_scaled)

# Split into train and test sets (80/20 split)
set.seed(42)
train_indices <- sample(1:nrow(X_scaled), size = 0.8 * nrow(X_scaled))
X_train <- X_scaled[train_indices, ]
X_test <- X_scaled[-train_indices, ]
y_train <- y[train_indices]
y_test <- y[-train_indices]

# Define the neural network structure
n_x <- ncol(X_train) # number of inputs
n_h <- 5             # neurons in hidden layer
n_y <- 1             # number of outputs

# Build the model
model <- keras_model_sequential() %>%
  layer_dense(units = n_h, activation = "relu", input_shape = n_x) %>%
  layer_dense(units = n_y, activation = "sigmoid")

# Define the neural network structure
n_x <- ncol(X_train) # number of inputs
n_h <- 5             # neurons in hidden layer
n_y <- 1             # number of outputs

# Build the model
model <- keras_model_sequential() %>%
  layer_dense(units = n_h, activation = "relu", input_shape = n_x) %>%
  layer_dense(units = n_y, activation = "sigmoid")

# Compile the model
model %>% compile(
  optimizer = "adam",
  loss = "binary_crossentropy",
  metrics = "accuracy"
)

# Train the model
history <- model %>% fit(
  X_train,
  y_train,
  epochs = 100,
  verbose = 0
)

# Compile the model
model %>% compile(
  optimizer = "adam",
  loss = "binary_crossentropy",
  metrics = "accuracy"
)

# Train the model
history <- model %>% fit(
  X_train,
  y_train,
  epochs = 100,
  verbose = 0
)

# Evaluate the model's loss on the training data
final_loss <- history$metrics$loss[length(history$metrics$loss)]
cat("Train Loss: ", final_loss, "\n") 

## Output:  Train Loss:  0.4451157

# Predict probabilities on test data
y_pred_prob <- model %>% predict(X_test)

# Convert probabilities to binary predictions using 0.5 threshold
y_pred <- ifelse(y_pred_prob > 0.5, 1, 0)

# Calculate accuracy
accuracy <- mean(y_pred == y_test)
cat("Accuracy: ", accuracy, "\n")
## Output: Accuracy:  0.95

# Predict probabilities on test data
y_pred_prob <- model %>% predict(X_test)

# Convert probabilities to binary predictions using 0.5 threshold
y_pred <- ifelse(y_pred_prob > 0.5, 1, 0)

# Calculate accuracy
accuracy <- mean(y_pred == y_test)
cat("Accuracy: ", accuracy, "\n")
## Output: Accuracy:  0.95

# Create a mesh grid over the feature space
x_min <- min(X_scaled[, 1]) - 0.5
x_max <- max(X_scaled[, 1]) + 0.5
y_min <- min(X_scaled[, 2]) - 0.5
y_max <- max(X_scaled[, 2]) + 0.5

grid <- expand.grid(
  Sepal.Length = seq(x_min, x_max, length.out = 200),
  Sepal.Width = seq(y_min, y_max, length.out = 200)
)

# Predict class probabilities for each point in the grid
grid_pred_prob <- model %>% predict(as.matrix(grid))
grid$Class <- as.factor(ifelse(grid_pred_prob > 0.5, "Versicolor", "Setosa"))

# Prepare the original data for plotting
plot_data <- data.frame(
  Sepal.Length = X_scaled[, 1],
  Sepal.Width = X_scaled[, 2],
  Species = factor(ifelse(y == 1, "Versicolor", "Setosa"))
)

# Plot decision boundary and data points
ggplot() +
  geom_contour_filled(
    data = grid,
    aes(x = Sepal.Length, y = Sepal.Width, z = as.numeric(Class == "Versicolor")),
    breaks = c(-Inf, 0.5, Inf),
    alpha = 0.4
  ) +
  geom_point(
    data = plot_data,
    aes(x = Sepal.Length, y = Sepal.Width, color = Species),
    size = 2
  ) +
  scale_color_manual(values = c("Setosa" = "red", "Versicolor" = "blue")) +
  labs(
    title = "Iris Classification Decision Boundary (Setosa=0, Versicolor=1)",
    x = "Sepal Length (scaled)",
    y = "Sepal Width (scaled)"
  ) +
  theme_bw()

# Create a mesh grid over the feature space
x_min <- min(X_scaled[, 1]) - 0.5
x_max <- max(X_scaled[, 1]) + 0.5
y_min <- min(X_scaled[, 2]) - 0.5
y_max <- max(X_scaled[, 2]) + 0.5

grid <- expand.grid(
  Sepal.Length = seq(x_min, x_max, length.out = 200),
  Sepal.Width = seq(y_min, y_max, length.out = 200)
)

# Predict class probabilities for each point in the grid
grid_pred_prob <- model %>% predict(as.matrix(grid))
grid$Class <- as.factor(ifelse(grid_pred_prob > 0.5, "Versicolor", "Setosa"))

# Prepare the original data for plotting
plot_data <- data.frame(
  Sepal.Length = X_scaled[, 1],
  Sepal.Width = X_scaled[, 2],
  Species = factor(ifelse(y == 1, "Versicolor", "Setosa"))
)

# Plot decision boundary and data points
ggplot() +
  geom_contour_filled(
    data = grid,
    aes(x = Sepal.Length, y = Sepal.Width, z = as.numeric(Class == "Versicolor")),
    breaks = c(-Inf, 0.5, Inf),
    alpha = 0.4
  ) +
  geom_point(
    data = plot_data,
    aes(x = Sepal.Length, y = Sepal.Width, color = Species),
    size = 2
  ) +
  scale_color_manual(values = c("Setosa" = "red", "Versicolor" = "blue")) +
  labs(
    title = "Iris Classification Decision Boundary (Setosa=0, Versicolor=1)",
    x = "Sepal Length (scaled)",
    y = "Sepal Width (scaled)"
  ) +
  theme_bw()

# Scale the inputs to (0, 1) range
X_scaled <- scale(X, center = TRUE, scale = TRUE)
X_scaled <- as.matrix(X_scaled)

# Split into train and test sets (80/20 split)
set.seed(42)
train_indices <- sample(1:nrow(X_scaled), size = 0.8 * nrow(X_scaled))
X_train <- X_scaled[train_indices, ]
X_test <- X_scaled[-train_indices, ]
y_train <- y[train_indices]
y_test <- y[-train_indices]

# Define the neural network structure
n_x <- ncol(X_train) # number of inputs
n_h <- 5             # neurons in hidden layer
n_y <- 1             # number of outputs

# Build the model
model <- keras_model_sequential() %>%
  layer_dense(units = n_h, activation = "relu", input_shape = n_x) %>%
  layer_dense(units = n_y, activation = "sigmoid")

# Compile the model
model %>% compile(
  optimizer = "adam",
  loss = "binary_crossentropy",
  metrics = "accuracy"
)

# Train the model
history <- model %>% fit(
  X_train,
  y_train,
  epochs = 100,
  verbose = 0
)

# Evaluate the model's loss on the training data
final_loss <- history$metrics$loss[length(history$metrics$loss)]
cat("Train Loss: ", final_loss, "\n") 

## Output:  Train Loss:  0.4451157

# Predict probabilities on test data
y_pred_prob <- model %>% predict(X_test)

# Convert probabilities to binary predictions using 0.5 threshold
y_pred <- ifelse(y_pred_prob > 0.5, 1, 0)

# Calculate accuracy
accuracy <- mean(y_pred == y_test)
cat("Accuracy: ", accuracy, "\n")
## Output: Accuracy:  0.95

# Create a mesh grid over the feature space
x_min <- min(X_scaled[, 1]) - 0.5
x_max <- max(X_scaled[, 1]) + 0.5
y_min <- min(X_scaled[, 2]) - 0.5
y_max <- max(X_scaled[, 2]) + 0.5

grid <- expand.grid(
  Sepal.Length = seq(x_min, x_max, length.out = 200),
  Sepal.Width = seq(y_min, y_max, length.out = 200)
)

# Predict class probabilities for each point in the grid
grid_pred_prob <- model %>% predict(as.matrix(grid))
grid$Class <- as.factor(ifelse(grid_pred_prob > 0.5, "Versicolor", "Setosa"))

# Prepare the original data for plotting
plot_data <- data.frame(
  Sepal.Length = X_scaled[, 1],
  Sepal.Width = X_scaled[, 2],
  Species = factor(ifelse(y == 1, "Versicolor", "Setosa"))
)

# Plot decision boundary and data points
ggplot() +
  geom_contour_filled(
    data = grid,
    aes(x = Sepal.Length, y = Sepal.Width, z = as.numeric(Class == "Versicolor")),
    breaks = c(-Inf, 0.5, Inf),
    alpha = 0.4
  ) +
  geom_point(
    data = plot_data,
    aes(x = Sepal.Length, y = Sepal.Width, color = Species),
    size = 2
  ) +
  scale_color_manual(values = c("Setosa" = "red", "Versicolor" = "blue")) +
  labs(
    title = "Iris Classification Decision Boundary (Setosa=0, Versicolor=1)",
    x = "Sepal Length (scaled)",
    y = "Sepal Width (scaled)"
  ) +
  theme_bw()

Introduction to Neural Network Operations

Working with the Iris Dataset

Join the 1M+ learners on CodeSignal

Data Preprocessing and Splitting

Building a Simple Neural Network

Understanding Activation Functions

Other Common Activation Functions

Implementing Forward Propagation

Computing Cost Function

Model Training & Evaluation

Visualizing the Decision Boundary

Lesson Summary and Practice