From Brain Cells to the McCulloch–Pitts Neuron

Start here Every deep learning model, from a phone’s face unlock to a chatbot, is built from one tiny unit: the artificial neuron. It is a very simple copy of a brain cell. This page shows where the idea came from and builds the very first artificial neuron (from 1943) step by step.

The idea comes from biology — from the brain. A brain cell is called a neuron (also: neural cell, neural processing unit). Deep learning borrows the name and a very rough version of how it works.

Everyday example You see a funny video. Your eyes and ears (sense organs) send signals to neurons. Some neurons fire (become active). They pass the signal to other neurons, which may also fire … and finally you laugh.

One neuron does not do all this. Your brain has about $10^{11} = 100$ billion neurons in a huge, massively parallel network.

Two big ideas from the brain:

1. Division of work. Each neuron has its own job. In the lecture’s cartoon, one neuron fires “if the text is funny”, one “if the visual is funny”, one “if the speech style is funny”, and a top neuron fires “if at least 2 of these 3 fired”.
2. Hierarchy (layers). In the visual cortex, information flows from the retina through layers V1, V2, … up to AIT. Lower layers find simple things; higher layers find complex things.

Lecture example — face detection in layers Layer 1: detect edges and corners → Layer 2: group them into features like nose, mouth, eyes → Layer 3: detect a whole face.

The professor’s honest disclaimer: this is a very simplified story of how the brain works. It is enough for this course. Real brains are far more complex.

In one line: An artificial neuron is a tiny, simplified copy of a brain cell; many of them in layers make a neural network.

Warren McCulloch (a neuroscientist) and Walter Pitts (a logician) proposed the first, very simple, computational model of a neuron in 1943.

The rules:

• Inputs $x_{1}, x_{2}, \ldots , x_{n}$ are only 0 or 1 (Boolean).
• Output y is also only 0 or 1.
• Each input is either excitatory (normal) or inhibitory (a “veto” input).

Step 1 — g adds up the inputs:

$g(x) = x_{1} + x_{2} + \ldots + x_{n}$

Step 2 — f makes the decision:

• $y = 1$ if $g(x) \ge \theta$
• $y = 0$ if $g(x) < \theta$

$\theta$ (theta) is called the thresholding parameter. This whole idea is called thresholding logic.

Veto rule: if any inhibitory input is 1, the output is 0, no matter what.

Everyday example You go to a cricket match only if at least 2 of these are true: friend is coming $(x_{1})$, it is a weekend $(x_{2})$, tickets are cheap $(x_{3})$. $So \theta = 2$. But if “I have an exam tomorrow” (inhibitory) is 1, you do not go at all.

Lecture example — check OR with numbers ($\theta = 1, 2$ inputs) $(0,0): 0 + 0 = 0$, and $0 < 1 \to y = 0 (0,1): 0 + 1 = 1$, and $1 \ge 1 \to y = 1 (1,0): 1 + 0 = 1 \ge 1 \to y = 1 (1,1): 1 + 1 = 2 \ge 1 \to y = 1$ ✔ this is OR

With 2 inputs there are only 4 possible points: (0,0), (0,1), (1,0), (1,1). The MP neuron fires when $x_{1} + x_{2} \ge \theta$. The boundary $x_{1} + x_{2} = \theta$ is a straight line.

• OR: line $x_{1} + x_{2} = 1$. Only (0,0) is below it.
• AND: line $x_{1} + x_{2} = 2$. Only (1,1) is on/above it.
• With 3 inputs, the line becomes a plane (for $OR: x_{1} + x_{2} + x_{3} = 1$ separates (0,0,0) from the other 7 points). With n inputs it is a hyperplane.

Linearly separable (for Boolean functions): there is a line (plane) such that every input with output 1 is on one side and every input with output 0 is on the other side.

In one line: A single MP neuron can represent only linearly separable Boolean functions.

The perceptron (next page) fixes the first three. A network of perceptrons fixes the fourth.

Exam words: artificial neuron, dendrite, soma, axon, synapse, McCulloch–Pitts neuron, excitatory input, inhibitory input, thresholding logic, threshold $\theta$, linearly separable, hyperplane.