Stacking Layers: Building a Multi-Layer Perceptron (MLP)

Stacking Layers: Building a Multi-Layer Perceptron (MLP) | CodeSignal Learn

Introduction

Welcome to the first lesson of "The MLP Architecture: Activations & Initialization"! I'm excited to continue our neural network journey with you. In our previous course, Neural Network Fundamentals: Neurons and Layers, we built the foundations of neural networks by implementing individual neurons, adding activation functions, and combining neurons into a single DenseLayer capable of forward propagation. Today, we're taking a significant step forward by learning how to stack multiple layers together to create a Multi-Layer Perceptron (MLP) . MLPs are the fundamental architecture behind many neural network applications and represent the point where our implementations truly become "deep learning." By the end of this lesson, you'll have created a fully functional MLP capable of processing data through multiple layers, bringing us much closer to solving real-world problems. Let's dive in!

Recap: Our Neural Network Building Blocks

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Creating the MLP Class

Now that we understand the concept, let's start implementing our MLP . First, we'll create the basic class structure that will house our layers: Python class MLP: def __init__(self): """Initialize a Multi-Layer Perceptron.""" self.layers = [] class MLP: def __init__(self): """Initialize a Multi-Layer Perceptron.""" self.layers = [] This simple initialization creates an empty list that will store our layers. The key idea here is that our MLP will be a container for multiple DenseLayer objects arranged in sequence. Notice how we're deliberately keeping the initialization straightforward. The MLP doesn't need to know in advance how many layers it will contain or their dimensions — this flexibility lets us dynamically build networks of different architectures as needed. This design approach mirrors professional deep learning frameworks, which also allow for flexible network construction.

Adding Layers to the MLP

Next, we need a way to add layers to our MLP . Let's implement the add_layer method: Python def add_layer(self, layer): """Add a layer to the MLP.""" self.layers.append(layer) def add_layer(self, layer): """Add a layer to the MLP.""" self.layers.append(layer) This method is elegantly simple — it takes a layer object (which will be an instance of our previously created DenseLayer class) and appends it to our layers list. The beauty of this approach is its flexibility: We can add as many layers as we need Each layer can have different numbers of neurons We could potentially extend this to support different types of layers in the future When using this method, we'll need to ensure that the dimensions of consecutive layers match correctly — the number of outputs from one layer must equal the number of inputs to the next layer. This dimensional compatibility is essential for data to flow properly through the network.

Forward Propagation Through Multiple Layers

Now for the most crucial part: implementing forward propagation through all the layers in our MLP . This is where we'll see how the output of one layer becomes the input to the next: Python def forward(self, inputs): """ Perform a forward pass through all layers in the MLP. inputs: Input data for the first layer. """ current_input = inputs for layer in self.layers: current_input = layer.forward(current_input) return current_input def forward(self, inputs): """ Perform a forward pass through all layers in the MLP. inputs: Input data for the first layer. """ current_input = inputs for layer in self.layers: current_input = layer.forward(current_input) return current_input Let's break down what happens here: We initialize current_input with the original input data. We iterate through each layer in our network. For each layer, we: Call the layer's forward method with the current input. Update current_input with the output from that layer. After processing through all layers, we return the final output. This sequential processing is the essence of how information flows through an MLP . Each layer transforms the data, gradually shaping it into the desired output. The variable current_input serves as the "baton" in this relay race, carrying information from one layer to the next. The elegance of this approach is that the MLP doesn't need to know the internal details of each layer — it simply calls the forward method, trusting each layer to do its job correctly. This encapsulation is a powerful software design principle that allows us to build complex systems from simpler components.

Building an MLP Network

Now that we have our MLP class defined, let's see how to create a complete multi-layer perceptron with multiple dense layers: Python# Import necessary modules import numpy as np # Create a sample input X_sample = np.array([[1.0, 0.5, -1.0, 2.0]]) # Shape (1, 4) print(f"Input X (shape {X_sample.shape}):\n{X_sample}") # Create the MLP mlp = MLP() mlp.add_layer(DenseLayer(n_inputs=4, n_neurons=5)) # First layer mlp.add_layer(DenseLayer(n_inputs=5, n_neurons=3)) # Hidden layer mlp.add_layer(DenseLayer(n_inputs=3, n_neurons=1)) # Output layer # Print information about the MLP print(f"\nMLP created with {len(mlp.layers)} layers.") for i, layer in enumerate(mlp.layers): print(f" Layer {i+1}: {layer.n_inputs} inputs, {layer.n_neurons} neurons, Activation: {layer.activation_fn_name}")# Import necessary modules import numpy as np # Create a sample input X_sample = np.array([[1.0, 0.5, -1.0, 2.0]]) # Shape (1, 4) print(f"Input X (shape {X_sample.shape}):\n{X_sample}") # Create the MLP mlp = MLP() mlp.add_layer(DenseLayer(n_inputs=4, n_neurons=5)) # First layer mlp.add_layer(DenseLayer(n_inputs=5, n_neurons=3)) # Hidden layer mlp.add_layer(DenseLayer(n_inputs=3, n_neurons=1)) # Output layer # Print information about the MLP print(f"\nMLP created with {len(mlp.layers)} layers.") for i, layer in enumerate(mlp.layers): print(f" Layer {i+1}: {layer.n_inputs} inputs, {layer.n_neurons} neurons, Activation: {layer.activation_fn_name}") In this code, we: Create a sample input X_sample with 4 features (a single sample for now). Instantiate our MLP. Add three layers: The first layer takes 4 inputs (matching our input data) and produces 5 outputs. The second layer takes those 5 inputs and produces 3 outputs. The final layer takes 3 inputs and produces a single output. Print information about our constructed network. Notice how we've chained the layers together, ensuring that the number of inputs to each layer matches the number of outputs from the previous layer. This forms a coherent network where data can flow smoothly from input to output. The output shows: Input X (shape (1, 4)): [[ 1. 0.5 -1. 2. ]] MLP created with 3 layers. Layer 1: 4 inputs, 5 neurons, Activation: sigmoid Layer 2: 5 inputs, 3 neurons, Activation: sigmoid Layer 3: 3 inputs, 1 neurons, Activation: sigmoidInput X (shape (1, 4)): [[ 1. 0.5 -1. 2. ]] MLP created with 3 layers. Layer 1: 4 inputs, 5 neurons, Activation: sigmoid Layer 2: 5 inputs, 3 neurons, Activation: sigmoid Layer 3: 3 inputs, 1 neurons, Activation: sigmoid This gives us a clear picture of our network's architecture — a 3-layer MLP with a decreasing number of neurons in each layer, funneling down to a single output neuron.

Processing Data Through the MLP

Now let's run our input data through the MLP and examine the output: Python# Perform forward pass through all layers output = mlp.forward(X_sample) print(f"\nOutput of the MLP (shape {output.shape}):\n{output}") # Create a batch of inputs X_batch = np.array([ [1.0, 0.5, -1.0, 2.0], # First sample [0.1, -0.2, 0.3, -0.4] # Second sample ]) # Shape (2, 4) print(f"\nInput Batch X (shape {X_batch.shape}):\n{X_batch}") # Process the batch output_batch = mlp.forward(X_batch) print(f"\nOutput of the MLP for batch (shape {output_batch.shape}):\n{output_batch}")# Perform forward pass through all layers output = mlp.forward(X_sample) print(f"\nOutput of the MLP (shape {output.shape}):\n{output}") # Create a batch of inputs X_batch = np.array([ [1.0, 0.5, -1.0, 2.0], # First sample [0.1, -0.2, 0.3, -0.4] # Second sample ]) # Shape (2, 4) print(f"\nInput Batch X (shape {X_batch.shape}):\n{X_batch}") # Process the batch output_batch = mlp.forward(X_batch) print(f"\nOutput of the MLP for batch (shape {output_batch.shape}):\n{output_batch}") In this code: We perform a forward pass with our single sample input and print the result. We create a batch of 2 samples, each with 4 features. We run a forward pass with the batch and print the result. The output shows: Output of the MLP (shape (1, 1)): [[0.51571291]] Input Batch X (shape (2, 4)): [[ 1. 0.5 -1. 2. ] [ 0.1 -0.2 0.3 -0.4]] Output of the MLP for batch (shape (2, 1)): [[0.51571291] [0.51563099]]Output of the MLP (shape (1, 1)): [[0.51571291]] Input Batch X (shape (2, 4)): [[ 1. 0.5 -1. 2. ] [ 0.1 -0.2 0.3 -0.4]] Output of the MLP for batch (shape (2, 1)): [[0.51571291] [0.51563099]] Several important observations: Our single sample input produced a single scalar output (wrapped in a 2D array to maintain batch structure). Our batch of 2 samples produced 2 outputs — one for each sample. The output values are different for each sample, showing that our network processes each sample individually. All outputs are in the range (0, 1) because we're using the sigmoid activation function in all layers. This confirms that our MLP is working correctly! It can process both individual samples and batches of data, maintaining the correct output dimensions throughout the network.

Introduction

Recap: Our Neural Network Building Blocks

Join the 1M+ learners on CodeSignal

Understanding Multi-Layer Perceptrons

Creating the MLP Class

Adding Layers to the MLP

Forward Propagation Through Multiple Layers

Building an MLP Network

Processing Data Through the MLP

Conclusion and Next Steps