The Perceptron

Introduction

The perceptron is the simplest form of a neural network, invented by Frank Rosenblatt in 1957. It's a linear binary classifier that forms the foundation for understanding more complex neural networks.

How It Works

A perceptron takes multiple inputs, applies weights to them, adds a bias, and produces a binary output:

output = sign(w₁×x₁ + w₂×x₂ + ... + wₙ×xₙ + bias)

The perceptron learning algorithm adjusts weights based on mistakes:

  1. Initialize weights randomly
  2. For each training example:
    • Calculate the prediction
    • If incorrect, update weights: w = w + η × error × input
  3. Repeat until convergence or maximum epochs

Interactive Perceptron Trainer

Click to add blue points (class +1), shift-click for red points (class -1). Watch how the perceptron learns to separate them:

Click: add blue point (+1) | Shift+Click: add red point (-1)

Controls

Current Weights

w₁ = 0.500
w₂ = -0.300
bias = 0.100

Visual Indicators:

  • ● Yellow circles: Misclassified points
  • — Purple line: Decision boundary
  • → Green arrow: Weight update direction
  • Bottom text: Current errors (and best found)

Limitations

The perceptron can only learn linearly separable patterns. This means it can only separate classes that can be divided by a straight line (or hyperplane in higher dimensions).

Famous example: The XOR problem cannot be solved by a single perceptron because XOR is not linearly separable. This limitation led to the development of multi-layer perceptrons (MLPs).

Key Takeaways

  • The perceptron is a linear binary classifier
  • It learns by adjusting weights based on prediction errors
  • Guaranteed to converge if data is linearly separable
  • Forms the building block for more complex neural networks
  • Understanding the perceptron helps grasp concepts like weights, bias, and gradient-based learning