Data Preparation for Neural Networks

Introduction

Welcome back to lesson 4 of "Building and Applying Your Neural Network Library"! You’ve made great progress so far. We started by modularizing our core components for layers and activations, then organized our training pipeline with modular loss functions and optimizers. Most recently, we built a high-level orchestration layer with our Model and SequentialModel classes, providing a clean API for building and training neural networks. Now it’s time to put our complete neural network library to work on a real-world problem! While our XOR examples were perfect for learning, real machine learning applications involve working with actual datasets that come with their own challenges: multiple features, varying scales, missing values, and the need for proper data preprocessing. In this lesson, we’ll prepare a California Housing dataset sample — a classic regression problem that predicts house prices based on various demographic and geographic features. We’ll learn essential data handling techniques, including loading real datasets, understanding feature characteristics, splitting data properly for evaluation, and applying feature scaling to ensure our neural network can learn effectively. This foundation will set us up perfectly for our final lesson, where we’ll apply our complete neural network library to solve this practical prediction problem.

Understanding Real-World Datasets

Real datasets come with complexities that require careful handling before we can apply machine learning algorithms effectively: Feature diversity is one key challenge — real datasets often contain features measured in completely different units and scales. For example, our California Housing dataset includes features like median income, house age, average rooms per household, and geographic coordinates. These dramatically different scales can cause problems for neural networks, which work best when all inputs are in similar ranges. Data splitting becomes crucial when working with real datasets. Unlike our toy examples, where we could evaluate on the same data we trained on, real applications require us to reserve some data for testing. This allows us to get an honest estimate of how our model will perform on new, unseen data — the true test of machine learning success. Preprocessing requirements also become more sophisticated. We need to standardize features so they have similar scales, handle the train-test split properly to avoid data leakage, and ensure our preprocessing steps are applied consistently between training and testing phases.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Loading the California Housing Dataset

For this lesson, you will be provided with the California Housing dataset in CSV format, located at data/california_housing.csv. This file contains several records, each with multiple features and a target value representing the median house value.

To load and parse the CSV file, you can use the PapaParse library, which is a popular and easy-to-use CSV parser for JavaScript. Here’s how you can use PapaParse to load the dataset, extract features and targets, and inspect the data:

JavaScript

import * as math from 'mathjs';
import seedrandom from 'seedrandom';
import fs from 'fs';
import Papa from 'papaparse';

// 1. Load the California Housing dataset from CSV file
const csv = fs.readFileSync('data/california_housing.csv', 'utf8');

// 2. Parse the CSV file using PapaParse
const parsed = Papa.parse(csv, {
    header: true,           // Treat the first row as column headers
    dynamicTyping: true,    // Automatically convert values to numbers when possible
    skipEmptyLines: true    // Ignore empty lines in the CSV
});

// 3. Extract feature names and target name from the header
const allColumns = Object.keys(parsed.data[0]);
const featureNames = allColumns.slice(0, -1); // All columns except the last are features
const targetName = allColumns[allColumns.length - 1]; // The last column is the target

// 4. Convert parsed data into feature matrix X and target vector y
const X = [];
const y = [];
parsed.data.forEach(row => {
    // For each row, collect feature values in order
    X.push(featureNames.map(f => row[f]));
    // Collect the target value as a single-element array (column vector)
    y.push([row[targetName]]);
});

// 5. Print out shapes and sample data for inspection
console.log(`Raw data shapes: X=[${X.length}, ${X[0].length}], y=[${y.length}, ${y[0].length}]`);
console.log(`Number of features: ${featureNames}`);
console.log(`Target name: MedHouseVal (Median House Value in $100,000s)`);
console.log(`Sample X[0]: [${X[0].map(x => x.toFixed(2)).join(', ')}]`);
console.log(`Sample y[0]: [${y[0][0].toFixed(2)}]`);

Example Output:

text

Raw data shapes: X=[5, 8], y=[5, 1]
Number of features: MedInc,HouseAge,AveRooms,AveBedrms,Population,AveOccup,Latitude,Longitude
Target name: MedHouseVal (Median House Value in $100,000s)
Sample X[0]: [8.33, 41.00, 6.98, 1.02, 322.00, 2.56, 37.88, -122.23]
Sample y[0]: [4.53]

We have several housing records with multiple features each. The features include median income (MedInc), house age (HouseAge), average rooms per household (AveRooms), and geographic coordinates (Latitude, Longitude). Notice the dramatic scale differences: median income ranges around 3–8, house age is in years (21–52), while geographic coordinates are around 37–38 for latitude and -122 for longitude. The target variable (MedHouseVal) represents median house values in units of $100,000.

Splitting Data for Proper Evaluation

To evaluate our model honestly, we need to split our data into training and testing sets. We can shuffle the data and then slice it into two parts. For reproducibility, we’ll use a simple seeded random shuffle.

Here’s a basic implementation:

JavaScript

function trainTestSplit(X, y, testSize = 0.2, randomState = 42) {
    const random = seedrandom(randomState.toString());
    const nSamples = X.length;
    const nTest = Math.floor(nSamples * testSize);
    
    // Create indices and shuffle them
    const indices = Array.from({ length: nSamples }, (_, i) => i);
    for (let i = indices.length - 1; i > 0; i--) {
        const j = Math.floor(random() * (i + 1));
        [indices[i], indices[j]] = [indices[j], indices[i]];
    }
    
    const testIndices = indices.slice(0, nTest);
    const trainIndices = indices.slice(nTest);
    
    const XTrain = trainIndices.map(i => X[i]);
    const XTest = testIndices.map(i => X[i]);
    const yTrain = trainIndices.map(i => y[i]);
    const yTest = testIndices.map(i => y[i]);
    
    return { XTrain, XTest, yTrain, yTest };
}

// 2. Split data into training and testing sets
const { XTrain, XTest, yTrain, yTest } = trainTestSplit(X, y, 0.2, 42);
console.log(`\nData split shapes:`);
console.log(`  XTrain: [${XTrain.length}, ${XTrain[0].length}], yTrain: [${yTrain.length}, ${yTrain[0].length}]`);
console.log(`  XTest: [${XTest.length}, ${XTest[0].length}], yTest: [${yTest.length}, ${yTest[0].length}]`);

Example Output:

text

Data split shapes:
  XTrain: [4, 8], yTrain: [4, 1]
  XTest: [1, 8], yTest: [1, 1]

We’ve allocated 80% of our data for training and 20% for testing. The randomState parameter ensures reproducible results — we’ll get the same split every time we run the code. This consistency is important for comparing different models or hyperparameters fairly.

The key principle here is that our model will never see the test data during training. This separation allows us to evaluate how well our neural network generalizes to new, unseen examples.

Feature Scaling for Neural Networks

Neural networks are particularly sensitive to the scale of input features. When features have very different ranges, the network may have difficulty learning effectively. Let’s apply standardization (z-score normalization) to transform our features.

Here’s a standard scaler implementation:

JavaScript

function standardScaler(data) {
    const matrix = math.matrix(data);
    const mean = math.mean(matrix, 0);
    const std = math.std(matrix, 0);
    
    const scaled = math.subtract(matrix, mean);
    const result = math.dotDivide(scaled, std);
    
    return {
        data: result.toArray(),
        mean: mean.toArray(),
        std: std.toArray(),
        transform: function(newData) {
            const newMatrix = math.matrix(newData);
            const scaledNew = math.subtract(newMatrix, mean);
            return math.dotDivide(scaledNew, std).toArray();
        }
    };
}

function standardScaler(data) {
    const matrix = math.matrix(data);
    const mean = math.mean(matrix, 0);
    const std = math.std(matrix, 0);
    
    const scaled = math.subtract(matrix, mean);
    const result = math.dotDivide(scaled, std);
    
    return {
        data: result.toArray(),
        mean: mean.toArray(),
        std: std.toArray(),
        transform: function(newData) {
            const newMatrix = math.matrix(newData);
            const scaledNew = math.subtract(newMatrix, mean);
            return math.dotDivide(scaledNew, std).toArray();
        }
    };
}

We’ll compute the mean and standard deviation for each feature using only the training data, then use these statistics to scale both the training and test sets. This avoids data leakage.

JavaScript

// 3. Apply feature scaling (Standardization)
// Initialize scaler
const scalerX = standardScaler(XTrain);
// Fit scaler on training data and transform training data
const XTrainScaled = scalerX.data;
// Transform test data using the *same* scaler fitted on training data
const XTestScaled = scalerX.transform(XTest);

// Scale target variable y as well (often beneficial for regression)
const scalerY = standardScaler(yTrain);
const yTrainScaled = scalerY.data;
const yTestScaled = scalerY.transform(yTest); // Using scalerY fitted on yTrain

// 3. Apply feature scaling (Standardization)
// Initialize scaler
const scalerX = standardScaler(XTrain);
// Fit scaler on training data and transform training data
const XTrainScaled = scalerX.data;
// Transform test data using the *same* scaler fitted on training data
const XTestScaled = scalerX.transform(XTest);

// Scale target variable y as well (often beneficial for regression)
const scalerY = standardScaler(yTrain);
const yTrainScaled = scalerY.data;
const yTestScaled = scalerY.transform(yTest); // Using scalerY fitted on yTrain

Notice the crucial distinction: we use only the training data to compute the means and standard deviations, and then apply those statistics to both training and test sets. This ensures we don’t introduce data leakage — the test data statistics don’t influence our preprocessing parameters.

Verifying Our Preprocessing

Let’s verify that our scaling worked correctly by examining the statistical properties of our transformed data:

JavaScript

console.log("\n--- After Scaling ---");
const XTrainMatrix = math.matrix(XTrainScaled);
const meanScaled = math.mean(XTrainMatrix, 0).toArray().map(x => x.toFixed(2));
const stdScaled = math.std(XTrainMatrix, 0).toArray().map(x => x.toFixed(2));

console.log(`XTrainScaled mean (should be close to 0): [${meanScaled.join(', ')}]`);
console.log(`XTrainScaled std (should be close to 1): [${stdScaled.join(', ')}]`);
console.log(`Sample XTrainScaled[0]: [${XTrainScaled[0].map(x => x.toFixed(2)).join(', ')}]`);

const yTrainMatrix = math.matrix(yTrainScaled);
console.log(`\nyTrainScaled mean: ${math.mean(yTrainMatrix).toFixed(2)}`);
console.log(`yTrainScaled std: ${math.std(yTrainMatrix).toFixed(2)}`);
console.log(`Sample yTrainScaled[0]: [${yTrainScaled[0][0].toFixed(2)}]`);

console.log("\n--- After Scaling ---");
const XTrainMatrix = math.matrix(XTrainScaled);
const meanScaled = math.mean(XTrainMatrix, 0).toArray().map(x => x.toFixed(2));
const stdScaled = math.std(XTrainMatrix, 0).toArray().map(x => x.toFixed(2));

console.log(`XTrainScaled mean (should be close to 0): [${meanScaled.join(', ')}]`);
console.log(`XTrainScaled std (should be close to 1): [${stdScaled.join(', ')}]`);
console.log(`Sample XTrainScaled[0]: [${XTrainScaled[0].map(x => x.toFixed(2)).join(', ')}]`);

const yTrainMatrix = math.matrix(yTrainScaled);
console.log(`\nyTrainScaled mean: ${math.mean(yTrainMatrix).toFixed(2)}`);
console.log(`yTrainScaled std: ${math.std(yTrainMatrix).toFixed(2)}`);
console.log(`Sample yTrainScaled[0]: [${yTrainScaled[0][0].toFixed(2)}]`);

Example Output:

text

--- After Scaling ---
XTrainScaled mean (should be close to 0): [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00]
XTrainScaled std (should be close to 1): [1.00, 1.00, 1.00, 1.00, 1.00, 1.00, 1.00, 1.00]
Sample XTrainScaled[0]: [0.44, 0.13, 1.18, 0.36, -0.89, 0.38, 0.00, 0.00]

yTrainScaled mean: 0.00
yTrainScaled std: 1.00
Sample yTrainScaled[0]: [1.41]

--- After Scaling ---
XTrainScaled mean (should be close to 0): [0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00]
XTrainScaled std (should be close to 1): [1.00, 1.00, 1.00, 1.00, 1.00, 1.00, 1.00, 1.00]
Sample XTrainScaled[0]: [0.44, 0.13, 1.18, 0.36, -0.89, 0.38, 0.00, 0.00]

yTrainScaled mean: 0.00
yTrainScaled std: 1.00
Sample yTrainScaled[0]: [1.41]

Perfect! Our scaled features now have means approximately 0 and standard deviations of 1 across all dimensions. The sample that originally had values ranging from 1.02 to 2401 now has standardized values in a much narrower range. Similarly, our target variable has been scaled to have zero mean and unit variance.

This transformation puts all our features on equal footing, allowing our neural network to learn effectively without being dominated by features that happen to have larger numerical values. The network can now focus on the actual patterns and relationships in the data rather than fighting against scale differences.

Data Preparation for Neural Networks

Introduction

Understanding Real-World Datasets

Join the 1M+ learners on CodeSignal

Loading the California Housing Dataset

Splitting Data for Proper Evaluation

Feature Scaling for Neural Networks

Verifying Our Preprocessing

Conclusion and Next Steps