Model Orchestration in R

Introduction

Welcome back to the third lesson of "Building and Applying Your Neural Network Library"! You've made tremendous progress in this course. In our first lesson, we successfully modularized our core neural network components — dense layers and activation functions . Then, in our previous lesson, we organized our training components by creating dedicated modules for loss functions and optimizers, building a solid foundation for our neural network package. However, as you may have noticed from our previous training examples, we're still writing quite a bit of boilerplate code for each training session. We manually create layers, set up optimizers, write training loops, handle forward and backward passes, and coordinate all these components ourselves. While this gives us complete control, it also means we're repeating the same orchestration logic every time we want to train a model. In this lesson, we're going to orchestrate all these components into a unified, high-level interface. We'll build a powerful Model R6 class that acts as the conductor of our neural network orchestra, coordinating layers, optimizers, and loss functions through clean, intuitive methods like compile(), fit(), and predict(), as well as a SequentialModel concrete subclass that can be seen as a better and improved replacement for any manual training approach we developed previously, providing a much more elegant and maintainable API for building and training neural networks. Let's get started!

The Need for Orchestration

Think of a symphony orchestra — while each musician is skilled at playing their individual instrument, the magic happens when a conductor coordinates all these talents toward a unified performance. Similarly, we've built excellent individual components (layers, optimizers, losses), but we need a conductor to orchestrate them into a seamless training experience. Currently, our training process requires us to manually coordinate several moving parts: Instantiate layers and build our network architecture. Create an optimizer with specific parameters. Define our loss function. Implement the training loop with forward passes, loss calculations, backward passes, and weight updates. This manual orchestration is error-prone and repetitive — exactly the kind of work that should be automated. What we need is a model class that serves as this conductor, providing a high-level API that handles the complexities of training coordination while still giving us the flexibility to customize our network architecture, choose different optimizers and loss functions, and control training parameters. R # Current goal: elegant and extensible training API model <- SequentialModel$new() model$add(DenseLayer$new(n_inputs = 2, n_neurons = 4, activation_fn_name = "relu")) model$compile(optimizer_name = "sgd", learning_rate = 0.5, loss_name = "mse") model$fit(X_train, y_train, epochs = 1000)

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Implementing Training and Prediction Logic

Now let's add the core training and prediction functionality. The virtual method pattern we're using ensures scalability — any new model architecture can plug into our training orchestration system by simply implementing the required methods.

Model <- R6Class("Model",
  public = list(
    layers = NULL,
    optimizer = NULL,
    loss_fn = NULL,
    loss_fn_derivative = NULL,
    is_compiled = NULL,
    
    initialize = function() {
      self$layers <- list()
      self$optimizer <- NULL
      self$loss_fn <- NULL
      self$loss_fn_derivative <- NULL
      self$is_compiled <- FALSE
    },
    
    compile = function(optimizer_name = 'sgd', learning_rate = 0.01, loss_name = 'mse') {
      # Set up the optimizer - easily extensible for new optimizers
      if (tolower(optimizer_name) == 'sgd') {
        self$optimizer <- SGD$new(learning_rate = learning_rate)
      } else {
        stop(paste("Unsupported optimizer:", optimizer_name))
      }
      
      # Set up the loss function - easily extensible for new loss functions
      if (tolower(loss_name) == 'mse') {
        self$loss_fn <- mse_loss
        self$loss_fn_derivative <- mse_loss_derivative
      } else {
        stop(paste("Unsupported loss function:", loss_name))
      }
      
      self$is_compiled <- TRUE
      cat(sprintf("Model compiled with optimizer: %s (lr: %s), loss: %s\n", 
                  optimizer_name, learning_rate, loss_name))
    },
    
    .forward = function(inputs) {
      # Virtual method - must be implemented by subclasses
      stop("_forward method must be implemented by subclasses")
    },
    
    .backward = function(d_loss_wrt_prediction) {
      # Virtual method - must be implemented by subclasses  
      stop("_backward method must be implemented by subclasses")
    },
    
    predict = function(X) {
      # Generates output predictions for the input samples
      return(self$.forward(X))
    },
    
    fit = function(X_train, y_train, epochs, batch_size = NULL, verbose = 1) {
      # Trains the model for a fixed number of epochs with automatic orchestration
      if (!self$is_compiled) {
        stop("Model must be compiled before training. Call model$compile()")
      }
      
      num_samples <- nrow(X_train)
      if (is.null(batch_size)) {
        batch_size <- num_samples  # Full batch gradient descent
      }
      
      for (epoch in 1:epochs) {
        epoch_loss <- 0.0
        # Shuffle data each epoch for better convergence
        indices <- sample(1:num_samples)
        
        # Process data in batches
        for (i in seq(1, num_samples, batch_size)) {
          end_idx <- min(i + batch_size - 1, num_samples)
          batch_indices <- indices[i:end_idx]
          X_batch <- X_train[batch_indices, , drop = FALSE]
          y_batch <- y_train[batch_indices, , drop = FALSE]
          
          # Coordinated forward-backward-update cycle
          y_pred <- self$.forward(X_batch)                        # Forward pass
          loss <- self$loss_fn(y_batch, y_pred)                  # Loss calculation
          epoch_loss <- epoch_loss + loss * nrow(X_batch)        # Accumulate loss
          
          d_loss_wrt_pred <- self$loss_fn_derivative(y_batch, y_pred)  # Loss gradient
          self$.backward(d_loss_wrt_pred)                         # Backward pass
          
          # Update all trainable layers
          for (layer in self$layers) {
            if (!is.null(layer$weights) && !is.null(layer$d_weights)) {
              self$optimizer$update(layer)
            }
          }
        }
        
        # Progress reporting
        epoch_loss <- epoch_loss / num_samples
        if (verbose > 0) {
          # Print progress, e.g., every 10% of epochs or at least the first and last
          if (epochs < 10 || (epoch %% max(1, epochs %/% 10) == 0) || epoch == 1) {
            cat(sprintf("Epoch %4d/%d, Loss: %.6f\n", epoch, epochs, epoch_loss))
          }
        }
      }
      
      if (verbose > 0) {
        cat("Training finished.\n")
      }
    }
  )
)

The fit() method orchestrates the entire training process with several key responsibilities:

Validation: Ensures the model is properly compiled before training.
Batch processing: Handles flexible batch sizes for memory efficiency and training stability.
Data management: Shuffles data each epoch to prevent overfitting to data order.
Training coordination: Manages the forward-backward-update cycle automatically.
Progress tracking: Provides informative training progress feedback.
Architecture independence: Works with any model architecture through the virtual method pattern, with .forward and .backward methods.

This design ensures our training system can scale to handle different model types, training strategies, and dataset sizes without requiring code changes to the core training logic.

Building the Sequential Model

Now we can create our concrete SequentialModel class that implements the virtual methods from our base Model. This class represents the familiar stack-of-layers architecture we've been working with, but now with a much cleaner interface and full integration into our extensible framework.

SequentialModel <- R6Class("SequentialModel",
  inherit = Model,
  public = list(
    initialize = function(layers_list = NULL) {
      # Initialize a sequential model, optionally with a pre-built layer list
      super$initialize()  # Initialize base Model attributes
      if (!is.null(layers_list)) {
        for (layer in layers_list) {
          self$add(layer)
        }
      }
    },
    
    add = function(layer) {
      # Add a layer instance to the sequential stack
      self$layers[[length(self$layers) + 1]] <- layer
    },
    
    .forward = function(inputs) {
      # Forward pass: data flows sequentially through all layers
      current_input <- inputs
      for (layer in self$layers) {
        current_input <- layer$forward(current_input)
      }
      return(current_input)
    },
    
    .backward = function(d_loss_wrt_prediction) {
      # Backward pass: gradients flow in reverse through all layers
      current_d_loss <- d_loss_wrt_prediction
      for (i in length(self$layers):1) {
        current_d_loss <- self$layers[[i]]$backward(current_d_loss)
      }
    }
  )
)

The SequentialModel demonstrates excellent separation of concerns:

Initialization flexibility: Supports both empty initialization and pre-built layer lists.
Architecture building: The add() method provides the familiar interface for building networks layer by layer.
Sequential processing: The forward pass flows data through layers in order, and the backward pass propagates gradients in reverse.
Clean responsibility: Focuses solely on managing a linear sequence of layers while inheriting all training orchestration from the parent class.

This design makes our library highly extensible — we can easily add other model types, like convolutional networks or attention mechanisms, by implementing the same virtual interface, while all the training logic remains reusable.

Introduction

The Need for Orchestration

Join the 1M+ learners on CodeSignal

Understanding Abstract Classes in R6

Designing the Base Model Class

Implementing Training and Prediction Logic

Building the Sequential Model

Using Our Orchestrated Model

Discussing the Output

Conclusion