Model Orchestration Patterns

Introduction

Welcome back to the third lesson of "Building and Applying Your Neural Network Library"! You've already made great progress modularizing your neural network codebase. In lesson 1, you organized your core components — dense layers and activation functions — using a modern JavaScript project structure. In lesson 2, you separated your training components by creating dedicated modules for loss functions and optimizers. Now, your codebase is clean, maintainable, and ready for the next step. However, as you may have noticed, our training scripts still involve a lot of repetitive code. Every time we want to train a model, we have to manually create layers, set up the optimizer, define the loss function, write the training loop, and coordinate all these parts ourselves. This is not only tedious but also error-prone. In this lesson, you'll learn how to orchestrate all these components into a unified, high-level interface. We'll build a powerful Model base class that acts as the conductor of our neural network, coordinating layers, optimizers, and loss functions through clean, intuitive methods like compile(), fit(), and predict() . We'll also implement a SequentialModel subclass that provides a much more elegant and maintainable API for building and training neural networks. Let's get started!

The Need for Orchestration

Think of a symphony orchestra: each musician is skilled at their instrument, but the magic happens when a conductor brings them together for a unified performance. Similarly, we've built excellent individual components (layers, optimizers, losses), but we need a conductor to orchestrate them into a seamless training experience. Currently, our training process requires us to manually coordinate several moving parts: Instantiating layers and building the network architecture Creating an optimizer with specific parameters Defining the loss function Implementing the training loop with forward passes, loss calculations, backward passes, and weight updates This manual orchestration is repetitive and error-prone. What we need is a Model class that serves as the conductor, providing a high-level API that handles the complexities of training while still giving us the flexibility to customize our network architecture, choose different optimizers and loss functions, and control training parameters. Here's the kind of interface we're aiming for: JavaScript const model = new SequentialModel(); model.add(new DenseLayer(...)); model.compile({ optimizerName: 'sgd', learningRate: 0.5, lossName: 'mse' }); model.fit(XTrain, yTrain, { epochs: 1000 });

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Designing the Base Model Class

Let's implement our Model orchestrator. This class will define the common interface and shared functionality for all types of neural networks. In JavaScript, we don't have abstract classes, but we can still design a base class that expects certain methods to be implemented by subclasses. To make this expectation clear and to help catch mistakes early, we'll provide default implementations of _forward and _backward in the base class that throw an error if they're not overridden. This way, if a subclass forgets to implement these methods, you'll get a clear error message at runtime.

neuralnets/models/model.js

JavaScript

import * as math from 'mathjs';
import { SGD } from '../optimizers/sgd.js';
import { mseLoss, mseLossDerivative } from '../losses/mse.js';

export class Model {
    constructor() {
        this.layers = [];              // Layer stack - populated by subclasses
        this.optimizer = null;         // Training optimizer
        this.lossFn = null;            // Loss function
        this.lossFnDerivative = null;  // Loss derivative function
        this.isCompiled = false;       // Compilation status flag
    }

    compile({ optimizerName = 'sgd', learningRate = 0.01, lossName = 'mse' } = {}) {
        // Set up the optimizer - easily extensible for new optimizers
        if (optimizerName.toLowerCase() === 'sgd') {
            this.optimizer = new SGD(learningRate);
        } else {
            throw new Error(`Unsupported optimizer: ${optimizerName}`);
        }

        // Set up the loss function - easily extensible for new loss functions
        if (lossName.toLowerCase() === 'mse') {
            this.lossFn = mseLoss;
            this.lossFnDerivative = mseLossDerivative;
        } else {
            throw new Error(`Unsupported loss function: ${lossName}`);
        }
        
        this.isCompiled = true;
        console.log(
            `Model compiled with optimizer: ${optimizerName} (lr: ${learningRate}), loss: ${lossName}`
        );
    }

    // These methods must be implemented by subclasses
    _forward(inputs) {
        throw new Error('_forward() must be implemented by the subclass');
    }

    _backward(dLossWrtPrediction) {
        throw new Error('_backward() must be implemented by the subclass');
    }
}

This foundation establishes all the essential components our model will need to coordinate:

Layer stack management: The layers array holds our network architecture
Optimizer coordination: Centralized optimizer configuration and management
Loss function setup: Unified loss function and derivative handling
Compilation validation: The isCompiled flag prevents training misconfiguration
Extensibility: The string-based configuration system makes adding new optimizers and loss functions straightforward
Error signaling: The base _forward and _backward methods throw errors to ensure subclasses implement them

Implementing Training and Prediction Logic

Now let's add the core training and prediction functionality. The base class provides predict and fit methods that rely on the subclass implementations of _forward and _backward. If these methods are not implemented, the error-throwing base methods will make the problem obvious.

neuralnets/models/model.js (continued)

export class Model {
    // ... previous code ...

    predict(X) {
        // Generates output predictions for the input samples
        return this._forward(X);
    }

    fit(XTrain, yTrain, { epochs = 100, batchSize = null, verbose = 1 } = {}) {
        if (!this.isCompiled) {
            throw new Error("Model must be compiled before training. Call model.compile().");
        }
        
        const numSamples = XTrain.size()[0];
        const actualBatchSize = batchSize || numSamples; // Full batch if not specified

        for (let epoch = 0; epoch < epochs; epoch++) {
            let epochLoss = 0.0;

            // Shuffle data each epoch for better convergence
            const indices = Array.from({ length: numSamples }, (_, i) => i);
            for (let i = indices.length - 1; i > 0; i--) {
                const j = Math.floor(Math.random() * (i + 1));
                [indices[i], indices[j]] = [indices[j], indices[i]];
            }

            // Process data in batches
            for (let i = 0; i < numSamples; i += actualBatchSize) {
                const batchIndices = indices.slice(i, i + actualBatchSize);
                const XBatch = math.subset(XTrain, math.index(batchIndices, math.range(0, XTrain.size()[1])));
                const yBatch = math.subset(yTrain, math.index(batchIndices, math.range(0, yTrain.size()[1])));

                // Coordinated forward-backward-update cycle
                const yPred = this._forward(XBatch);                              // Forward pass
                const loss = this.lossFn(yBatch, yPred);                         // Loss calculation
                epochLoss += loss * XBatch.size()[0];                             // Accumulate loss

                const dLossWrtPred = this.lossFnDerivative(yBatch, yPred);       // Loss gradient
                this._backward(dLossWrtPred);                                      // Backward pass

                // Update all trainable layers
                for (const layer of this.layers) {
                    if (
                        layer.weights !== undefined && layer.dWeights !== null &&
                        layer.biases !== undefined && layer.dBiases !== null
                    ) {
                        this.optimizer.update(layer);
                    }
                }
            }

            // Progress reporting
            epochLoss /= numSamples;
            
            if (verbose) {
                if ((epochs < 10) || ((epoch + 1) % Math.max(1, Math.floor(epochs / 10)) === 0) || epoch === 0) {
                    console.log(
                        `Epoch ${(epoch + 1).toString().padStart(4)}/${epochs}, Loss: ${epochLoss.toFixed(6)}`
                    );
                }
            }
        }
        
        if (verbose) {
            console.log("Training finished.");
        }
    }
}

The fit() method orchestrates the entire training process with several key responsibilities:

Validation: Ensures the model is properly compiled before training
Batch processing: Handles flexible batch sizes for memory efficiency and training stability
Data management: Shuffles data each epoch to prevent overfitting to data order
Training coordination: Manages the forward-backward-update cycle automatically
Progress tracking: Provides informative training progress feedback
Architecture independence: Works with any model architecture, as long as the subclass provides _forward and _backward methods (otherwise, a clear error is thrown)

This design ensures our training system can scale to handle different model types, training strategies, and dataset sizes without requiring code changes to the core training logic.

Building the Sequential Model

Now we can create our concrete SequentialModel class that implements the expected methods from our base Model. This class represents the familiar stack-of-layers architecture we've been working with, but now with a much cleaner interface and full integration into our extensible framework.

neuralnets/models/sequential.js

JavaScript

import { Model } from './model.js';

export class SequentialModel extends Model {
    constructor(layersList = null) {
        super();
        if (layersList) {
            for (const layer of layersList) {
                this.add(layer);
            }
        }
    }

    add(layer) {
        this.layers.push(layer);
    }

    _forward(inputs) {
        let currentInput = inputs;
        for (const layer of this.layers) {
            currentInput = layer.forward(currentInput);
        }
        return currentInput;
    }

    _backward(dLossWrtPrediction) {
        let currentDLoss = dLossWrtPrediction;
        for (let i = this.layers.length - 1; i >= 0; i--) {
            currentDLoss = this.layers[i].backward(currentDLoss);
        }
    }
}

import { Model } from './model.js';

export class SequentialModel extends Model {
    constructor(layersList = null) {
        super();
        if (layersList) {
            for (const layer of layersList) {
                this.add(layer);
            }
        }
    }

    add(layer) {
        this.layers.push(layer);
    }

    _forward(inputs) {
        let currentInput = inputs;
        for (const layer of this.layers) {
            currentInput = layer.forward(currentInput);
        }
        return currentInput;
    }

    _backward(dLossWrtPrediction) {
        let currentDLoss = dLossWrtPrediction;
        for (let i = this.layers.length - 1; i >= 0; i--) {
            currentDLoss = this.layers[i].backward(currentDLoss);
        }
    }
}

The SequentialModel demonstrates excellent separation of concerns:

Initialization flexibility: Supports both empty initialization and pre-built layer lists
Architecture building: The add() method provides the familiar interface for building networks layer by layer
Sequential processing: The forward pass flows data through layers in order, and the backward pass propagates gradients in reverse
Clean responsibility: Focuses solely on managing a linear sequence of layers while inheriting all training orchestration from the parent class

This design makes our library highly extensible — we can easily add other model types by implementing the expected interface, while all the training logic remains reusable.

Using Our Orchestrated Model

Now let's see our orchestration in action! Here's how we can solve the XOR problem using our new high-level API, which is much cleaner and more maintainable than our previous manual approach. Notice how this same interface will work seamlessly when we extend our library with new layers, optimizers, or loss functions.

neuralnets/main.js

JavaScript

import * as math from 'mathjs';
import { DenseLayer } from './layers/index.js';
import { SequentialModel } from './models/index.js';

// Sample data: XOR problem
const XTrain = math.matrix([[0,0], [0,1], [1,0], [1,1]]);
const yTrain = math.matrix([[0], [1], [1], [0]]);

// Build model using SequentialModel - easily extensible to new layer types
const model = new SequentialModel();
model.add(new DenseLayer(2, 4, 'relu', 'random_scaled', 0.1));
model.add(new DenseLayer(4, 1, 'sigmoid', 'random_scaled', 0.1));

// Compile the model - ready for new optimizers and loss functions
model.compile({ optimizerName: 'sgd', learningRate: 0.5, lossName: 'mse' });

// Train the model - same interface works for any model architecture
console.log("\nTraining model for XOR problem...");
model.fit(XTrain, yTrain, { epochs: 1000, batchSize: XTrain.size()[0], verbose: 1 });

// Test the trained model
console.log("\n--- After Training ---");
const finalPreds = model.predict(XTrain);
console.log("Input  | True Output | Predicted Output | Rounded Prediction");
for (let i = 0; i < XTrain.size()[0]; i++) {
    const input = math.subset(XTrain, math.index(i, math.range(0, 2)));
    const trueOutput = math.subset(yTrain, math.index(i, 0));
    const predVal = math.subset(finalPreds, math.index(i, 0));
    const roundedPred = Math.round(predVal);
    console.log(`${input._data} | ${trueOutput.toString().padStart(11)} | ${predVal.toFixed(4).padStart(16)} | ${roundedPred.toString().padStart(18)}`);
}

Notice how much cleaner this is compared to our previous training scripts! The beauty of this approach is that it maintains the flexibility to customize every aspect of our network while eliminating the repetitive boilerplate code. The extensible design means we can easily experiment with different architectures, optimizers, or hyperparameters without rewriting the training logic each time.

Discussing the Output

When you run your orchestrated model, you should see output similar to the following:

text

Model compiled with optimizer: sgd (lr: 0.5), loss: mse

Training model for XOR problem...
Epoch    1/1000, Loss: 0.249842
Epoch  100/1000, Loss: 0.219619
Epoch  200/1000, Loss: 0.055226
Epoch  300/1000, Loss: 0.014176
Epoch  400/1000, Loss: 0.006978
Epoch  500/1000, Loss: 0.004460
Epoch  600/1000, Loss: 0.003217
Epoch  700/1000, Loss: 0.002481
Epoch  800/1000, Loss: 0.001689
Epoch  900/1000, Loss: 0.001444
Epoch 1000/1000, Loss: 0.001444
Training finished.

--- After Training ---
Input  | True Output | Predicted Output | Rounded Prediction
0,0 |           0 |           0.0471 |                  0
0,1 |           1 |           0.9742 |                  1
1,0 |           1 |           0.9743 |                  1
1,1 |           0 |           0.0471 |                  0

Model compiled with optimizer: sgd (lr: 0.5), loss: mse

Training model for XOR problem...
Epoch    1/1000, Loss: 0.249842
Epoch  100/1000, Loss: 0.219619
Epoch  200/1000, Loss: 0.055226
Epoch  300/1000, Loss: 0.014176
Epoch  400/1000, Loss: 0.006978
Epoch  500/1000, Loss: 0.004460
Epoch  600/1000, Loss: 0.003217
Epoch  700/1000, Loss: 0.002481
Epoch  800/1000, Loss: 0.001689
Epoch  900/1000, Loss: 0.001444
Epoch 1000/1000, Loss: 0.001444
Training finished.

--- After Training ---
Input  | True Output | Predicted Output | Rounded Prediction
0,0 |           0 |           0.0471 |                  0
0,1 |           1 |           0.9742 |                  1
1,0 |           1 |           0.9743 |                  1
1,1 |           0 |           0.0471 |                  0

The results demonstrate excellent performance — our orchestrated model achieves the same learning quality as our previous manual implementations, but with much cleaner, more maintainable code. The loss decreases smoothly from 0.25 to 0.0014, and the final predictions correctly solve the XOR problem with high confidence. More importantly, the training process is now fully automated, consistently implemented, and ready to scale to more complex problems and architectures.

Model Orchestration Patterns

Introduction

The Need for Orchestration

Join the 1M+ learners on CodeSignal

Designing the Base Model Class

Implementing Training and Prediction Logic

Building the Sequential Model

Using Our Orchestrated Model

Discussing the Output

Conclusion