Implementing Forward Propagation in a Dense Layer with JavaScript

Introduction

Welcome back to our course "Neural Network Fundamentals: Neurons and Layers"! You're making excellent progress. In the previous units, we built a single artificial neuron, enhanced it with the sigmoid activation function, and then combined multiple neurons into a DenseLayer class using JavaScript and mathjs . In this final lesson, we'll bring our dense layer to life by implementing forward propagation — the process by which information travels through a neural network from input to output. This is where our layer actually processes data and produces meaningful activations. By the end of this lesson, you'll have a functional dense layer that can take inputs, process them through multiple neurons simultaneously, and produce outputs that could be fed to another layer or used directly for predictions.

Understanding Forward Propagation

Before we dive into code, let's clarify what forward propagation means in the context of neural networks. Forward propagation (or forward pass) is the process of taking input data and passing it through the network to generate an output. It's called "forward" because information flows forward through the network, from the input layer, through any hidden layers, to the output layer. For our dense layer, forward propagation involves three key steps: Weighted sum calculation : Multiply each input feature by its corresponding weight. Bias addition : Add the bias term to each weighted sum. Activation : Apply the activation function to introduce non-linearity. This process transforms the input data into activations (outputs) that represent what the layer has "learned" about the input. These activations can then be used as inputs to subsequent layers in deeper networks.

Matrix Operations for Efficient Processing

As we learned in the previous units, we're using matrices to represent the weights and biases of our dense layer. This allows us to process multiple neurons simultaneously through efficient matrix operations using mathjs . Let's clarify what we mean by a sample : a sample is a unit of input that the model processes to produce a prediction. For example, suppose you're building a neural network to predict whether a person will buy a product based on 3 features: Age, Income, and Number of previous purchases (in that order). Then each sample would be one person’s data, like [25, 40000, 2], where 25 is the age, 40000 is the income, and 2 is the number of previous purchases. Let's review the key matrices involved in forward propagation: Inputs : Shape [nSamples, nInputs] — each row represents one data sample. Weights : Shape [nInputs, nNeurons] — each column represents the weights for one neuron. Biases : Shape [nNeurons] — one bias per neuron. Outputs : Shape [nSamples, nNeurons] — each row will be the output for one sample. The beauty of this approach is that it works not only for a single input sample but also for batches of samples. When we have multiple samples (a batch), we can process them all at once without writing additional code. For example, if we have a batch of 4 samples, each with 3 features, and our layer has 2 neurons: Inputs shape: [4, 3] (4 samples, each with 3 features) Weights shape: [3, 2] (3 inputs connected to 2 neurons) Result of matrix multiplication: [4, 2] (4 samples, each producing 2 outputs) This efficient batch processing is one of the key advantages of using matrix operations in neural networks.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Implementing the Forward Method

Now, let's implement the forward method in our DenseLayer class. We'll focus first on calculating the weighted sum using mathjs:

JavaScript

const math = require('mathjs');

/**
 * DenseLayer class: Represents a fully connected layer of neurons.
 */
class DenseLayer {
    /**
     * Initialize a dense layer.
     * @param {number} nInputs - Number of input features.
     * @param {number} nNeurons - Number of neurons in this layer.
     */
    constructor(nInputs, nNeurons) {
        // Weights matrix: shape [nInputs, nNeurons]
        this.weights = math.multiply(math.random([nInputs, nNeurons]), 0.1);

        // Biases matrix: shape [1, nNeurons]
        this.biases = math.zeros(1, nNeurons);

        this.nInputs = nInputs;
        this.nNeurons = nNeurons;
    }

    /**
     * Perform a forward pass through the dense layer.
     * @param {Array} inputs - Input data. Shape: [nSamples, nInputs]
     * @returns {Array} Output activations. Shape: [nSamples, nNeurons]
     */
    forward(inputs) {
        // Calculate weighted sum: inputs (dot) weights
        // inputs shape: [nSamples, nInputs]
        // this.weights shape: [nInputs, nNeurons]
        // weightedSum shape: [nSamples, nNeurons]
        const weightedSum = math.multiply(inputs, this.weights);

        // We'll add biases and apply activation in the next section
        // ...
    }
}

const math = require('mathjs');

/**
 * DenseLayer class: Represents a fully connected layer of neurons.
 */
class DenseLayer {
    /**
     * Initialize a dense layer.
     * @param {number} nInputs - Number of input features.
     * @param {number} nNeurons - Number of neurons in this layer.
     */
    constructor(nInputs, nNeurons) {
        // Weights matrix: shape [nInputs, nNeurons]
        this.weights = math.multiply(math.random([nInputs, nNeurons]), 0.1);

        // Biases matrix: shape [1, nNeurons]
        this.biases = math.zeros(1, nNeurons);

        this.nInputs = nInputs;
        this.nNeurons = nNeurons;
    }

    /**
     * Perform a forward pass through the dense layer.
     * @param {Array} inputs - Input data. Shape: [nSamples, nInputs]
     * @returns {Array} Output activations. Shape: [nSamples, nNeurons]
     */
    forward(inputs) {
        // Calculate weighted sum: inputs (dot) weights
        // inputs shape: [nSamples, nInputs]
        // this.weights shape: [nInputs, nNeurons]
        // weightedSum shape: [nSamples, nNeurons]
        const weightedSum = math.multiply(inputs, this.weights);

        // We'll add biases and apply activation in the next section
        // ...
    }
}

In this code:

The forward method takes an input array, which can contain multiple samples.
We use mathjs's math.multiply to perform matrix multiplication between the inputs and weights.
This operation computes the weighted sum for every neuron in the layer, for every sample in the batch.

The matrix multiplication creates a new matrix where each element is the dot product of a row from the inputs and a column from the weights. Each row in the result corresponds to a sample, and each column corresponds to a neuron.

Adding Biases and Applying Activation

After calculating the weighted sum, we need to add the biases and apply the activation function. Since our biases are now stored as a [1, nNeurons] matrix, we can use math.add to broadcast the biases across all rows of the weighted sum.

We'll also use the sigmoid activation function we defined earlier:

JavaScript

/**
 * Sigmoid activation function.
 * @param {Array} x - Input array or number.
 * @returns {Array} Output after applying sigmoid.
 */
function sigmoid(x) {
    // mathjs supports element-wise operations
    return math.map(x, v => 1 / (1 + Math.exp(-v)));
}

/**
 * Sigmoid activation function.
 * @param {Array} x - Input array or number.
 * @returns {Array} Output after applying sigmoid.
 */
function sigmoid(x) {
    // mathjs supports element-wise operations
    return math.map(x, v => 1 / (1 + Math.exp(-v)));
}

Now, let's complete the forward method:

JavaScript

    forward(inputs) {
        // Calculate weighted sum: [nSamples, nNeurons]
        const weightedSum = math.multiply(inputs, this.weights);

        // Add biases to each row of the weighted sum
        // weightedSum: [nSamples, nNeurons]
        // this.biases: [1, nNeurons]
        // math.add will broadcast biases across all rows
        const outputBeforeActivation = math.add(weightedSum, this.biases);

        // Apply activation function (sigmoid) element-wise
        this.output = sigmoid(outputBeforeActivation);

        return this.output;
    }

    forward(inputs) {
        // Calculate weighted sum: [nSamples, nNeurons]
        const weightedSum = math.multiply(inputs, this.weights);

        // Add biases to each row of the weighted sum
        // weightedSum: [nSamples, nNeurons]
        // this.biases: [1, nNeurons]
        // math.add will broadcast biases across all rows
        const outputBeforeActivation = math.add(weightedSum, this.biases);

        // Apply activation function (sigmoid) element-wise
        this.output = sigmoid(outputBeforeActivation);

        return this.output;
    }

Here's what happens in this code:

We add the biases to each row of the weighted sum. Since this.biases is a [1, nNeurons] matrix, math.add broadcasts it across all rows.
We apply the sigmoid activation function to the result. This transforms every value in the matrix to be between 0 and 1, introducing non-linearity.
We store the result in this.output and also return it, making it available for the next layer or for the final prediction.

This completes the forward propagation process for our dense layer. Each neuron in our layer has now processed the input data, and we have the resulting activations.

Introduction

Understanding Forward Propagation

Matrix Operations for Efficient Processing

Join the 1M+ learners on CodeSignal

Implementing the Forward Method

Adding Biases and Applying Activation

Processing Data in Batches

Visualizing the Forward Pass

Conclusion and Next Steps