Building a Perceptron Model

Welcome to our journey into the foundational building block of Neural Networks: the perceptron! This essential algorithm is a stepping stone to understanding more advanced neural network models used in Machine Learning. In this lesson, you will learn the structure of a perceptron, how it makes predictions, and how it can be trained. By the end, you will have implemented a fully functioning perceptron model in C++ that can solve a simple logical problem using the AND operator data.

Let’s begin by exploring the perceptron, a simple type of binary linear classifier in the neural network family. A perceptron takes multiple inputs, combines them, and decides the output based on these inputs.

You can think of perceptrons as a voting system. Each "voter" (input) has a different weight (importance). The "candidate" (output) is chosen if the total weighted votes (inputs) surpass a certain threshold.

Mathematically, the perceptron’s output can be described as:

Where:

• $y$ is the output we are predicting.
• $w$ represents the weights — the importance of each input.
• $x$ is the vector of inputs.
• $b$ is the bias — a constant that shifts the decision boundary.
• $f$ is the activation function — it takes the sum and produces the final output.

Let’s start by setting up our perceptron in C++. We will use a class with a constructor to initialize the perceptron’s parameters.

#include <vector>
#include <iostream>

class Perceptron {
public:
    Perceptron(int no_of_inputs, int max_iterations = 100, double learning_rate = 0.01)
        : max_iterations(max_iterations), learning_rate(learning_rate)
    {
        // Initialize weights (including bias) to zero
        weights = std::vector<double>(no_of_inputs + 1, 0.0);
    }

private:
    int max_iterations;
    double learning_rate;
    std::vector<double> weights;
};

Here:

• no_of_inputs is the number of inputs to the perceptron.
• max_iterations is the maximum number of times the model will update its weights during training.
• learning_rate controls how much the weights are adjusted during each update.
• weights is a vector of doubles, initialized to zero, with one extra element for the bias.

Now, let’s see how the perceptron makes predictions. We will implement a predict method that calculates the weighted sum of the inputs and applies a step activation function.

public:
    int predict(const std::vector<double>& inputs) {
        double summation = weights[0]; // bias
        for (size_t i = 0; i < inputs.size(); ++i) {
            summation += weights[i + 1] * inputs[i];
        }
        return (summation > 0) ? 1 : 0;
    }

• The method starts with the bias (weights[0]).
• It adds the product of each input and its corresponding weight.
• If the total is greater than zero, the output is 1; otherwise, it is 0.

Next, we need to train our perceptron so it can learn from data. The train method updates the weights based on the prediction error.

public:
    void train(const std::vector<std::vector<double>>& training_inputs, const std::vector<int>& labels) {
        for (int iter = 0; iter < max_iterations; ++iter) {
            for (size_t i = 0; i < training_inputs.size(); ++i) {
                int prediction = predict(training_inputs[i]);
                double error = labels[i] - prediction;
                // Update weights for inputs
                for (size_t j = 0; j < training_inputs[i].size(); ++j) {
                    weights[j + 1] += learning_rate * error * training_inputs[i][j];
                }
                // Update bias
                weights[0] += learning_rate * error;
            }
        }
    }

• For each iteration and for each training example, the perceptron predicts the output.
• The error is calculated as the difference between the actual label and the prediction.
• The weights and bias are updated to reduce this error.

Let’s put everything together and apply our perceptron to a simple logical problem: the AND operator. The AND operator outputs 1 only if both inputs are 1; otherwise, it outputs 0.

int main() {
    // Prepare training data for AND operator
    std::vector<std::vector<double>> training_inputs = {
        {0, 0},
        {0, 1},
        {1, 0},
        {1, 1}
    };
    std::vector<int> labels = {0, 0, 0, 1};

    // Create and train perceptron
    Perceptron perceptron(2);
    perceptron.train(training_inputs, labels);

    // Test the perceptron
    std::vector<double> test_input1 = {1, 1};
    std::cout << "Prediction for [1, 1]: " << perceptron.predict(test_input1) << std::endl;

    std::vector<double> test_input2 = {1, 0};
    std::cout << "Prediction for [1, 0]: " << perceptron.predict(test_input2) << std::endl;

    std::vector<double> test_input3 = {0, 0};
    std::cout << "Prediction for [0, 0]: " << perceptron.predict(test_input3) << std::endl;

    return 0;
}

• We define the training data and labels for the AND operator, using the standard truth table order.
• We create a perceptron with two inputs and train it.
• We test the perceptron with new inputs and print the results.

Congratulations! You have learned how to understand, design, and implement a perceptron using C++. Practicing these concepts will help solidify your understanding and prepare you for more advanced topics in machine learning. Continue experimenting with different logical operators and datasets to further develop your skills!