Implementing Forward Propagation in a Dense Layer with JavaScript

Introduction

Welcome back to our course "Neural Network Fundamentals: Neurons and Layers"! You're making excellent progress. In the previous units, we built a single artificial neuron, enhanced it with the sigmoid activation function, and then combined multiple neurons into a DenseLayer class using JavaScript and mathjs . In this final lesson, we'll bring our dense layer to life by implementing forward propagation — the process by which information travels through a neural network from input to output. This is where our layer actually processes data and produces meaningful activations. By the end of this lesson, you'll have a functional dense layer that can take inputs, process them through multiple neurons simultaneously, and produce outputs that could be fed to another layer or used directly for predictions.

Understanding Forward Propagation

Before we dive into code, let's clarify what forward propagation means in the context of neural networks. Forward propagation (or forward pass) is the process of taking input data and passing it through the network to generate an output. It's called "forward" because information flows forward through the network, from the input layer, through any hidden layers, to the output layer. For our dense layer, forward propagation involves three key steps: Weighted sum calculation : Multiply each input feature by its corresponding weight. Bias addition : Add the bias term to each weighted sum. Activation : Apply the activation function to introduce non-linearity. This process transforms the input data into activations (outputs) that represent what the layer has "learned" about the input. These activations can then be used as inputs to subsequent layers in deeper networks.

Matrix Operations for Efficient Processing

As we learned in the previous units, we're using matrices to represent the weights and biases of our dense layer. This allows us to process multiple neurons simultaneously through efficient matrix operations using mathjs . Let's clarify what we mean by a sample : a sample is a unit of input that the model processes to produce a prediction. For example, suppose you're building a neural network to predict whether a person will buy a product based on 3 features: Age, Income, and Number of previous purchases (in that order). Then each sample would be one person’s data, like [25, 40000, 2], where 25 is the age, 40000 is the income, and 2 is the number of previous purchases. Let's review the key matrices involved in forward propagation: Inputs : Shape [nSamples, nInputs] — each row represents one data sample. Weights : Shape [nInputs, nNeurons] — each column represents the weights for one neuron. Biases : Shape [nNeurons] — one bias per neuron. Outputs : Shape [nSamples, nNeurons] — each row will be the output for one sample. The beauty of this approach is that it works not only for a single input sample but also for batches of samples. When we have multiple samples (a batch), we can process them all at once without writing additional code. For example, if we have a batch of 4 samples, each with 3 features, and our layer has 2 neurons: Inputs shape: [4, 3] (4 samples, each with 3 features) Weights shape: [3, 2] (3 inputs connected to 2 neurons) Result of matrix multiplication: [4, 2] (4 samples, each producing 2 outputs) This efficient batch processing is one of the key advantages of using matrix operations in neural networks.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

JavaScript

const math = require('mathjs');

/**
 * DenseLayer class: Represents a fully connected layer of neurons.
 */
class DenseLayer {
    /**
     * Initialize a dense layer.
     * @param {number} nInputs - Number of input features.
     * @param {number} nNeurons - Number of neurons in this layer.
     */
    constructor(nInputs, nNeurons) {
        // Weights matrix: shape [nInputs, nNeurons]
        this.weights = math.multiply(math.random([nInputs, nNeurons]), 0.1);

        // Biases matrix: shape [1, nNeurons]
        this.biases = math.zeros(1, nNeurons);

        this.nInputs = nInputs;
        this.nNeurons = nNeurons;
    }

    /**
     * Perform a forward pass through the dense layer.
     * @param {Array} inputs - Input data. Shape: [nSamples, nInputs]
     * @returns {Array} Output activations. Shape: [nSamples, nNeurons]
     */
    forward(inputs) {
        // Calculate weighted sum: inputs (dot) weights
        // inputs shape: [nSamples, nInputs]
        // this.weights shape: [nInputs, nNeurons]
        // weightedSum shape: [nSamples, nNeurons]
        const weightedSum = math.multiply(inputs, this.weights);

        // We'll add biases and apply activation in the next section
        // ...
    }
}

const math = require('mathjs');

/**
 * DenseLayer class: Represents a fully connected layer of neurons.
 */
class DenseLayer {
    /**
     * Initialize a dense layer.
     * @param {number} nInputs - Number of input features.
     * @param {number} nNeurons - Number of neurons in this layer.
     */
    constructor(nInputs, nNeurons) {
        // Weights matrix: shape [nInputs, nNeurons]
        this.weights = math.multiply(math.random([nInputs, nNeurons]), 0.1);

        // Biases matrix: shape [1, nNeurons]
        this.biases = math.zeros(1, nNeurons);

        this.nInputs = nInputs;
        this.nNeurons = nNeurons;
    }

    /**
     * Perform a forward pass through the dense layer.
     * @param {Array} inputs - Input data. Shape: [nSamples, nInputs]
     * @returns {Array} Output activations. Shape: [nSamples, nNeurons]
     */
    forward(inputs) {
        // Calculate weighted sum: inputs (dot) weights
        // inputs shape: [nSamples, nInputs]
        // this.weights shape: [nInputs, nNeurons]
        // weightedSum shape: [nSamples, nNeurons]
        const weightedSum = math.multiply(inputs, this.weights);

        // We'll add biases and apply activation in the next section
        // ...
    }
}

JavaScript

/**
 * Sigmoid activation function.
 * @param {Array} x - Input array or number.
 * @returns {Array} Output after applying sigmoid.
 */
function sigmoid(x) {
    // mathjs supports element-wise operations
    return math.map(x, v => 1 / (1 + Math.exp(-v)));
}

/**
 * Sigmoid activation function.
 * @param {Array} x - Input array or number.
 * @returns {Array} Output after applying sigmoid.
 */
function sigmoid(x) {
    // mathjs supports element-wise operations
    return math.map(x, v => 1 / (1 + Math.exp(-v)));
}

JavaScript

    forward(inputs) {
        // Calculate weighted sum: [nSamples, nNeurons]
        const weightedSum = math.multiply(inputs, this.weights);

        // Add biases to each row of the weighted sum
        // weightedSum: [nSamples, nNeurons]
        // this.biases: [1, nNeurons]
        // math.add will broadcast biases across all rows
        const outputBeforeActivation = math.add(weightedSum, this.biases);

        // Apply activation function (sigmoid) element-wise
        this.output = sigmoid(outputBeforeActivation);

        return this.output;
    }

    forward(inputs) {
        // Calculate weighted sum: [nSamples, nNeurons]
        const weightedSum = math.multiply(inputs, this.weights);

        // Add biases to each row of the weighted sum
        // weightedSum: [nSamples, nNeurons]
        // this.biases: [1, nNeurons]
        // math.add will broadcast biases across all rows
        const outputBeforeActivation = math.add(weightedSum, this.biases);

        // Apply activation function (sigmoid) element-wise
        this.output = sigmoid(outputBeforeActivation);

        return this.output;
    }

JavaScript

// Sample input data: 2 samples, each with 3 features
// This represents a batch of 2 data points.
const sampleBatchInputs = [
    [1.0, 2.0, 3.0],  // Sample 1
    [0.5, 1.5, 2.5]   // Sample 2
];
console.log('\nInput batch data (shape [' + sampleBatchInputs.length + ', ' + sampleBatchInputs[0].length + ']):');
console.log(sampleBatchInputs);

// Create a dense layer with 3 inputs and 2 neurons
const layer = new DenseLayer(3, 2);

// Perform forward pass
const layerOutput = layer.forward(sampleBatchInputs);

console.log('\nLayer output (shape [' + layerOutput.length + ', ' + layerOutput[0].length + ']):');
console.log(layerOutput);

// Sample input data: 2 samples, each with 3 features
// This represents a batch of 2 data points.
const sampleBatchInputs = [
    [1.0, 2.0, 3.0],  // Sample 1
    [0.5, 1.5, 2.5]   // Sample 2
];
console.log('\nInput batch data (shape [' + sampleBatchInputs.length + ', ' + sampleBatchInputs[0].length + ']):');
console.log(sampleBatchInputs);

// Create a dense layer with 3 inputs and 2 neurons
const layer = new DenseLayer(3, 2);

// Perform forward pass
const layerOutput = layer.forward(sampleBatchInputs);

console.log('\nLayer output (shape [' + layerOutput.length + ', ' + layerOutput[0].length + ']):');
console.log(layerOutput);

JavaScript

const weights = [
    [0.08, 0.05],
    [0.01, 0.09],
    [0.07, 0.02]
]; // shape: [3, 2]

const weights = [
    [0.08, 0.05],
    [0.01, 0.09],
    [0.07, 0.02]
]; // shape: [3, 2]

JavaScript

// [1.0, 2.0, 3.0] dot weights
const input = [[1.0, 2.0, 3.0]];
const weightedSum = math.multiply(input, weights);
// Result: [[0.31, 0.29]]

// [1.0, 2.0, 3.0] dot weights
const input = [[1.0, 2.0, 3.0]];
const weightedSum = math.multiply(input, weights);
// Result: [[0.31, 0.29]]

1.0 * 0.08 + 2.0 * 0.01 + 3.0 * 0.07
= 0.08 + 0.02 + 0.21
= 0.31

1.0 * 0.08 + 2.0 * 0.01 + 3.0 * 0.07
= 0.08 + 0.02 + 0.21
= 0.31

1.0 * 0.05 + 2.0 * 0.09 + 3.0 * 0.02
= 0.05 + 0.18 + 0.06
= 0.29

1.0 * 0.05 + 2.0 * 0.09 + 3.0 * 0.02
= 0.05 + 0.18 + 0.06
= 0.29

JavaScript

const biases = math.zeros(1, 2); // [1, 2] shape
const outputBeforeActivation = math.add(weightedSum, biases); // [[0.31, 0.29]]

const biases = math.zeros(1, 2); // [1, 2] shape
const outputBeforeActivation = math.add(weightedSum, biases); // [[0.31, 0.29]]

JavaScript

const output = sigmoid(outputBeforeActivation); // [[0.577, 0.572]]

const output = sigmoid(outputBeforeActivation); // [[0.577, 0.572]]

JavaScript

const math = require('mathjs');

/**
 * DenseLayer class: Represents a fully connected layer of neurons.
 */
class DenseLayer {
    /**
     * Initialize a dense layer.
     * @param {number} nInputs - Number of input features.
     * @param {number} nNeurons - Number of neurons in this layer.
     */
    constructor(nInputs, nNeurons) {
        // Weights matrix: shape [nInputs, nNeurons]
        this.weights = math.multiply(math.random([nInputs, nNeurons]), 0.1);

        // Biases matrix: shape [1, nNeurons]
        this.biases = math.zeros(1, nNeurons);

        this.nInputs = nInputs;
        this.nNeurons = nNeurons;
    }

    /**
     * Perform a forward pass through the dense layer.
     * @param {Array} inputs - Input data. Shape: [nSamples, nInputs]
     * @returns {Array} Output activations. Shape: [nSamples, nNeurons]
     */
    forward(inputs) {
        // Calculate weighted sum: inputs (dot) weights
        // inputs shape: [nSamples, nInputs]
        // this.weights shape: [nInputs, nNeurons]
        // weightedSum shape: [nSamples, nNeurons]
        const weightedSum = math.multiply(inputs, this.weights);

        // We'll add biases and apply activation in the next section
        // ...
    }
}

JavaScript

/**
 * Sigmoid activation function.
 * @param {Array} x - Input array or number.
 * @returns {Array} Output after applying sigmoid.
 */
function sigmoid(x) {
    // mathjs supports element-wise operations
    return math.map(x, v => 1 / (1 + Math.exp(-v)));
}

JavaScript

    forward(inputs) {
        // Calculate weighted sum: [nSamples, nNeurons]
        const weightedSum = math.multiply(inputs, this.weights);

        // Add biases to each row of the weighted sum
        // weightedSum: [nSamples, nNeurons]
        // this.biases: [1, nNeurons]
        // math.add will broadcast biases across all rows
        const outputBeforeActivation = math.add(weightedSum, this.biases);

        // Apply activation function (sigmoid) element-wise
        this.output = sigmoid(outputBeforeActivation);

        return this.output;
    }

JavaScript

// Sample input data: 2 samples, each with 3 features
// This represents a batch of 2 data points.
const sampleBatchInputs = [
    [1.0, 2.0, 3.0],  // Sample 1
    [0.5, 1.5, 2.5]   // Sample 2
];
console.log('\nInput batch data (shape [' + sampleBatchInputs.length + ', ' + sampleBatchInputs[0].length + ']):');
console.log(sampleBatchInputs);

// Create a dense layer with 3 inputs and 2 neurons
const layer = new DenseLayer(3, 2);

// Perform forward pass
const layerOutput = layer.forward(sampleBatchInputs);

console.log('\nLayer output (shape [' + layerOutput.length + ', ' + layerOutput[0].length + ']):');
console.log(layerOutput);

JavaScript

const weights = [
    [0.08, 0.05],
    [0.01, 0.09],
    [0.07, 0.02]
]; // shape: [3, 2]

JavaScript

// [1.0, 2.0, 3.0] dot weights
const input = [[1.0, 2.0, 3.0]];
const weightedSum = math.multiply(input, weights);
// Result: [[0.31, 0.29]]

1.0 * 0.08 + 2.0 * 0.01 + 3.0 * 0.07
= 0.08 + 0.02 + 0.21
= 0.31

1.0 * 0.05 + 2.0 * 0.09 + 3.0 * 0.02
= 0.05 + 0.18 + 0.06
= 0.29

JavaScript

const biases = math.zeros(1, 2); // [1, 2] shape
const outputBeforeActivation = math.add(weightedSum, biases); // [[0.31, 0.29]]

JavaScript

const output = sigmoid(outputBeforeActivation); // [[0.577, 0.572]]