California Housing Regression

Introduction

Welcome to the final lesson of "Building and Applying Your Neural Network Library". Congratulations on making it this far — you've accomplished something truly remarkable. Over the course of this path, you've built a complete, modular neural network library from scratch, learning the inner workings of layers, activations, optimizers, loss functions, and the orchestration that brings them all together. You've also mastered the essential data preparation techniques needed for real-world machine learning applications.

Today, we're going to experience the incredible satisfaction of seeing all your hard work come together. We'll use our custom-built neural network library to tackle a real regression problem: predicting California housing prices. You'll see how the modular architecture you've carefully constructed makes it surprisingly straightforward to define complex neural networks, train them efficiently, and evaluate their performance on real data.

This lesson represents the culmination of your journey — the moment when theory meets practice, and your carefully crafted code proves its worth on a meaningful problem. Let's put your neural network library to the ultimate test!

Project Structure

neuralnets-project/
├── CMakeLists.txt
├── main.cpp
└── include/
    └── neuralnets/          ← library headers (include path root)
        ├── models/
        │   └── SequentialModel.h
        ├── layers/
        │   └── DenseLayer.h
        └── losses/
            └── Losses.h
└── src/
    ├── models/
    │   └── SequentialModel.cpp
    ├── layers/
    │   └── DenseLayer.cpp
    └── losses/
        └── Losses.cpp

You'll notice two neuralnets references — one as the project name at the root, and one as a folder inside include/. The include/neuralnets/ folder is the library's include root, which is why headers are written as "neuralnets/layers/DenseLayer.h" rather than just "layers/DenseLayer.h". This is a standard C++ library convention that namespaces your headers and avoids conflicts with other libraries. All new files in this lesson are in include/neuralnets/models/ and src/models/ — everything else was built in previous lessons.

Setting Up the Data

Let's start by including our components and setting up the data preprocessing pipeline. Since you mastered data preparation in the previous lesson, we'll handle this efficiently and effortlessly:

#include <iostream>
#include <vector>
#include <random>
#include <fstream>
#include <sstream>
#include <string>
#include <Eigen/Dense>
#include "neuralnets/models/SequentialModel.h"
#include "neuralnets/layers/DenseLayer.h"
#include "neuralnets/losses/Losses.h"
#include "neuralnets/preprocessing/StandardScaler.h"

using namespace Eigen;
using namespace neuralnets;

// Generate synthetic California housing data for demonstration
std::pair<MatrixXd, MatrixXd> generateHousingData(int n_samples = 20640) {
    std::random_device rd;
    std::mt19937 gen(42); // Fixed seed for reproducibility
    std::normal_distribution<double> normal(0.0, 1.0);
    std::uniform_real_distribution<double> uniform(0.0, 1.0);
    
    MatrixXd X(n_samples, 8); // 8 features like the original dataset
    VectorXd y(n_samples);
    
    // Generate synthetic features representing housing characteristics
    for (int i = 0; i < n_samples; ++i) {
        X(i, 0) = uniform(gen) * 15.0 + 32.5;  // Latitude-like
        X(i, 1) = uniform(gen) * 4.0 - 124.0;  // Longitude-like  
        X(i, 2) = uniform(gen) * 50.0 + 1.0;   // Housing age
        X(i, 3) = uniform(gen) * 35000.0 + 500.0; // Total rooms
        X(i, 4) = uniform(gen) * 6000.0 + 100.0;  // Total bedrooms
        X(i, 5) = uniform(gen) * 12000.0 + 500.0; // Population
        X(i, 6) = uniform(gen) * 4500.0 + 100.0;  // Households
        X(i, 7) = uniform(gen) * 12.0 + 0.5;      // Median income
        
        // Generate target (house value) based on features with some noise
        double price = 0.5 + 0.3 * X(i, 7) + 0.1 * (50.0 - X(i, 2)) / 50.0 + 
                      0.2 * normal(gen) * 0.5; // Add noise
        y(i) = std::max(0.1, std::min(5.0, price)); // Clamp between 0.1 and 5.0
    }
    
    return {X, y.reshaped(n_samples, 1)};
}

// Split data into training and test sets
std::tuple<MatrixXd, MatrixXd, MatrixXd, MatrixXd> trainTestSplit(
    const MatrixXd& X, const MatrixXd& y, double test_size = 0.2, int random_state = 42) {
    
    std::mt19937 gen(random_state);
    std::vector<int> indices(X.rows());
    std::iota(indices.begin(), indices.end(), 0);
    std::shuffle(indices.begin(), indices.end(), gen);
    
    int test_samples = static_cast<int>(X.rows() * test_size);
    int train_samples = X.rows() - test_samples;
    
    MatrixXd X_train(train_samples, X.cols());
    MatrixXd X_test(test_samples, X.cols());
    MatrixXd y_train(train_samples, y.cols());
    MatrixXd y_test(test_samples, y.cols());
    
    for (int i = 0; i < train_samples; ++i) {
        X_train.row(i) = X.row(indices[i]);
        y_train.row(i) = y.row(indices[i]);
    }
    
    for (int i = 0; i < test_samples; ++i) {
        X_test.row(i) = X.row(indices[train_samples + i]);
        y_test.row(i) = y.row(indices[train_samples + i]);
    }
    
    return {X_train, X_test, y_train, y_test};
}

int main() {
    auto [X, y] = generateHousingData();
    auto [X_train, X_test, y_train, y_test] = trainTestSplit(X, y, 0.2, 42);
    
    // Reusing StandardScaler
    // then apply the same transformation to test data to avoid data leakage
    preprocessing::StandardScaler scaler_X;
    MatrixXd X_train_scaled = scaler_X.fit_transform(X_train);
    MatrixXd X_test_scaled = scaler_X.transform(X_test);

    preprocessing::StandardScaler scaler_y;
    MatrixXd y_train_scaled = scaler_y.fit_transform(y_train);
    MatrixXd y_test_scaled = scaler_y.transform(y_test);

    // Retrieve mean and std for inverse transforming predictions later
    double mean_y = scaler_y.get_mean()(0);
    double std_y = scaler_y.get_std()(0);
    
    int num_features = X_train_scaled.cols();
    
    std::cout << "Data loaded and preprocessed successfully!" << std::endl;
    std::cout << "Training samples: " << X_train.rows() << std::endl;
    std::cout << "Test samples: " << X_test.rows() << std::endl;
    std::cout << "Features: " << num_features << std::endl;
Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal