Understanding Neural Networks: An Introduction to the Perceptron Algorithm

Welcome aboard our exploration of the building block of Neural Networks: the perceptron! This significant algorithm sparks your comprehension of advanced Neural Networks utilized in Machine Learning. The objectives of this lesson include mastering and coding a perceptron using Python from scratch. We will decipher the structure of a perceptron, the prediction method, and the training process. To conclude, we will design a fully functioning model that abstracts a simple logical scenario using the AND operator data.

We start by delving into the perceptron, a simple form of binary linear classifiers in the Neural Network family. A perceptron operates by accepting multiple inputs, aggregating them, and democratically deciding the output based on these inputs.

Think of perceptrons as a democratic process. Each "voter" (input) contributes with differing weights. The "candidate" (output) who secures the majority of votes (aggregate of weighted inputs) wins and is chosen.

Mathematically, the predicted output of a perceptron can be formulated as follows:

For our purposes:

• $y$ is our output, which we are predicting.
• $w$ represents weights — consider these as the importance accorded to each contributing voter.
• $x$ shows our inputs, i.e., voters.
• $b$ is called bias — it's akin to an incumbent's advantage, a prior tendency towards a particular party or candidate.
• $f$ is an activation function — this takes in the sum of all votes and outputs the election result.

Let's kickstart our algorithm by setting the stage in our Python perceptron class with the __init__ method.

def __init__(self, no_of_inputs, max_iterations=100, learning_rate=0.01):
    self.max_iterations = max_iterations
    self.learning_rate = learning_rate
    self.weights = np.zeros(no_of_inputs + 1)

Here:

no_of_inputs refers to the number of inputs.

max_iterations is the maximum number of iterations the model will hold.

learning_rate indicates how fast we want the weights to adapt or learn based on the outcomes.

The initialized weights are set to zero, with an additional weight for the incumbent known as bias.