Understanding Forward Propagation in Neural Networks

Hello, and welcome to today's exciting lesson! We will delve into the world of neural networks, focusing on a technique called forward propagation, or the data flow from input to output in a neural network.

Neural networks are a variety of machine learning models inspired by the human brain. They draw upon the idea of having neurons interconnected in a net-like structure to process and learn from information, similar to how our brain learns from the data fed into it by our senses. One basic and essential step in how a neural network processes and learns from information is termed as forward propagation.

As the name suggests, forward propagation involves moving forward through the network. Each node in the network gets inputs from the nodes in the previous layer, multiplies them with their weights, adds a bias, and then "fires" that through an activation function. The result is then passed on as input to the nodes in the next layer. This process is repeated layer after layer until we reach the output layer, giving us the predicted output.

But what if the predicted output is far from the actual result? That's when backpropagation comes into play. In simple terms, backpropagation is the method used to update the weights of our neural network based on error correction. The less the error, the better our model predictions.

The entity that quantifies the error between predicted and actual outputs is the loss function. To minimize this loss and hence the prediction error, we use optimization algorithms like gradient descent. In this lesson, we focus on understanding forward propagation, setting a solid foundation for learning more intricate neural network operations such as backpropagation in future lessons.

Now, let's get our hands on practical implementation. We'll use the Iris dataset for our demonstration:

import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Input
import random
import tensorflow as tf
random.seed(42)
tf.random.set_seed(42)

# Loading Iris dataset
iris = datasets.load_iris()
X = iris.data[:, :2]  # we only take the first two features.
y = iris.target

# We onlt take the first two classes
X = X[y != 2]
y = y[y != 2]

We start by loading the Iris dataset, which is a multivariate dataset introduced by the British statistician and biologist Ronald Fisher in 1936. It's a go-to dataset for any beginner because of its simplicity. It includes the sepal length, sepal width, petal length, and petal width of 150 iris flowers from three different species. However, for this task, we only consider the first two features, which are sepal length and sepal width. And, we convert the problem into a binary classification problem — by taking the two classes.

Next up, we should preprocess our data accordingly:

# Scaling the inputs to (0, 1) range
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)

# Splitting dataset into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.2, random_state=42)

In machine learning, scaling the inputs is a common practice. It helps our model to converge faster. So, we scale our inputs to be in the (0, 1) range using MinMaxScaler(). After that, we split our dataset into a training set (80% of the data) and a test set (20% of the data). The model will learn from the training data and evaluate its performance on the unseen test data.