Chapter 14 The predictive brain

8 minute read

Illustration for Chapter 14, The predictive brain

Locked inside a pitch-black, silent vault of bone, your brain never touches the outside world directly. No light ever shines on the visual cortex, and no sound wave ever vibrates the temporal lobe. All the brain ever receives are noisy streams of electrical spikes arriving along sensory cables from the body. All it can ever send back are streams of spikes travelling down motor nerves to muscles and glands.

Yet out of those Morse-code clicks inside a dark skull, you experience a vivid three-dimensional world of faces, melodies, and autumn leaves. How does an organ trapped behind bone reconstruct the hidden causes of its sensory signals?

In 1867, the German physicist and physiologist Hermann von Helmholtz proposed a daring answer. Perception, he argued, is not a passive camera imprint stamped onto the mind by the senses. Instead, perception is an act of unconscious inference in which the brain guesses what hidden scene best explains its sensory clues. Over the past three decades, neuroscientists and machine learning pioneers turned Helmholtz's insight into a mathematical theory of the brain. That framework, shaped by Geoffrey Hinton, Peter Dayan, Rajesh Rao, Dana Ballard, and Karl Friston, is predictive processing.

14.1Variational inference and the free energy principle

To see the problem the brain faces every millisecond, imagine walking through a dim forest at dusk. Out in the trees, a hidden state of the world exists, such as a crouching tiger. You cannot observe that state directly. Instead, light bounces off the forest and strikes your retinas, creating a noisy pattern of sensory observations.

To make sense of those retinal flashes, the brain must carry an internal generative model of how the world works. Denoting the hidden external state and incoming sensory signal in the joint probability distribution p(s,x)=p(s∣x) p(x)p(s, x) = p(s \mid x) \, p(x), the model weaves two factors together. The first factor, the prior probability of the hidden state, captures how likely various external causes are before fresh evidence arrives. The second factor, the sensory likelihood of the observation, predicts what retinal pattern a given hidden object would cast.

When a fresh sensory signal arrives from the retina, probability theory gives a single rule for inverting this causal process. That rule is Bayes' theorem, which updates the prior belief into the posterior belief given the sensory evidence.

p(x∣s)=p(s∣x) p(x)p(s)=p(s,x)∫Xp(s,x′) dx′p(x \mid s) = \frac{p(s \mid x) \, p(x)}{p(s)} = \frac{p(s, x)}{\int_{\mathcal{X}} p(s, x') \, dx'}

At first glance, Bayes' theorem looks simple enough to compute on the back of an envelope. The catch lies in the denominator, known as the marginal likelihood or model evidence. It measures the total probability of encountering the sensory input across every possible combination of hidden causes in the state space.

In the real world, the space of hidden states spans millions of continuous, nonlinear dimensions. Evaluating that integral directly requires summing over an astronomical tree of possibilities, a task that is computationally intractable. Imagine a deer in the forest that had to integrate over every possible leaf and shadow before fleeing a rustle. It would be eaten long before the integral finished.

How can a biological brain built from slow, noisy neurons approximate Bayesian inference without ever computing that impossible denominator? The answer comes from a method invented by Richard Feynman in statistical physics and adapted to neuroscience by Hinton and Friston. That method is variational Bayesian inference, which turns an impossible integration problem into a continuous downhill optimization problem.

Instead of calculating the true posterior directly, the brain maintains an internal probability distribution qμ(x)q_\mu(x). This internal recognition density is encoded by the current firing rates and synaptic states of the brain's own neurons. The brain adjusts its internal neural state until its approximate belief matches the true posterior as closely as possible. Information theory measures the discrepancy between these two distributions using the Kullback-Leibler divergence, which is zero when the two distributions match and strictly positive otherwise.

Of course, the brain cannot compute the divergence to the unknown posterior directly because the intractable marginal evidence still sits inside it. Yet when we substitute Bayes' rule into the Kullback-Leibler divergence and split the logarithm, the marginal evidence pulls cleanly outside the integral. Rearranging the resulting terms yields the central identity of Karl Friston's free energy principle.

F(μ,s)=Eqμ[ln⁡qμ(x)−ln⁡p(s,x)]=−ln⁡p(s)+DKL(qμ(x)∥p(x∣s))\mathcal{F}(\mu, s) = \mathbb{E}_{q_\mu}\left[ \ln q_\mu(x) - \ln p(s, x) \right] = -\ln p(s) + D_{\text{KL}}\left( q_\mu(x) \parallel p(x \mid s) \right)

The quantity on the left is called the variational free energy, and in machine learning its negative is known as the evidence lower bound. This single equation accomplishes three remarkable feats at once for any self-organizing mind.

First, variational free energy is completely computable by the physical brain. The expectation on the left side of the identity depends only on the brain's current internal belief and its internal generative model of sensations. The intractable marginal integral has vanished from the calculation entirely.

Second, variational free energy sets a strict upper bound on sensory surprise. In information theory, the negative logarithm of the sensory probability −ln⁡p(s)-\ln p(s) is called surprisal, and it measures how improbable a sensory state is for a given organism. For a fish, cool water has high probability and low surprisal, whereas dry sand has immense surprisal. Because the Kullback-Leibler divergence is never negative, variational free energy is always greater than or equal to surprisal.

To stay alive against thermodynamic decay, an organism must keep its bodily states within narrow physiological bounds. That means keeping average surprisal low. Since an organism cannot measure surprisal directly, it keeps surprise low by continuously minimizing its upper bound, the variational free energy.

Third, this single equation unites perception and action under one shared mathematical law. Ask how a living creature can drive variational free energy downward. There are two distinct knobs it can turn. It can change its internal neural beliefs, or it can move its body through muscle actions to change the sensory signals themselves.

When the organism holds its senses fixed and changes its internal neural activity by sliding downhill along the free energy gradient, the surprisal term stays constant. Every drop in variational free energy therefore drives the Kullback-Leibler divergence straight toward zero. By minimizing free energy with respect to internal neural states, neural activity automatically converges onto the exact Bayesian posterior belief. That process is perception.

Conversely, the organism can reduce free energy by moving its muscles, turning its eyes, or stepping into the shade. These actions alter incoming sensory signals until they match the states that the organism's prior model expects to encounter. When your blood sugar drops and triggers surprising signals from your gut, you do not merely accept starvation. You walk to the kitchen and eat, changing your sensory inputs until they match your homeostatic predictions. That process is active inference, where motor control is the physical fulfillment of descending predictions.

We can also split variational free energy into two philosophical pieces by factoring the generative model into prior and likelihood. One piece measures predictive inaccuracy, punishing beliefs that fail to explain incoming sensory data. The other piece measures model complexity, penalizing the brain whenever it twists its beliefs too far away from its prior knowledge to chase random noise. Minimizing variational free energy therefore builds William of Ockham's razor directly into every neural circuit. The brain always seeks the simplest internal explanation that accurately accounts for the evidence of the senses.

14.2Predictive coding in the neocortex

How do actual cortical columns built of pyramidal neurons and inhibitory interneurons run gradient descent on variational free energy? In 1999, Rajesh Rao and Dana Ballard showed how the hierarchical anatomy of the visual cortex implements this optimization. Karl Friston later generalized their circuit model across the entire cortex using the Laplace approximation.

Suppose the neocortex is organized as a hierarchy of processing levels above raw sensory input at level zero at the bottom. It climbs from primary sensory areas such as V1 up to association areas in the frontal and temporal lobes. At each cortical level, a neural population encodes a Gaussian belief centered on a mean expectation vector. Each higher cortical area predicts the activity at the area just below it through a nonlinear top-down mapping. The random fluctuations at that lower level have a precision matrix defined as the inverse of the noise variance.

Under this Gaussian assumption, the abstract integral for variational free energy simplifies dramatically. Up to constants, free energy becomes a sum across all cortical levels of squared prediction errors weighted by their precision matrices. At each step of the hierarchy, the raw prediction error measures the mismatch between what the lower cortical area registers and what the higher area predicted. Multiplying that mismatch by the precision matrix gives the precision-weighted prediction error ξ(i)=Π(i)(μ(i−1)−g(i)(μ(i)))\xi^{(i)} = \Pi^{(i)}(\mu^{(i-1)} - g^{(i)}(\mu^{(i)})) computed at the lower area.

Now let the neural representations evolve in time by flowing downhill along the gradient of variational free energy. Taking the derivative of the quadratic sum yields the dynamical equation of hierarchical predictive coding.

μ˙(i)=−∂F∂μ(i)=(∂g(i)∂μ(i))Tξ(i)−ξ(i+1)\dot{\mu}^{(i)} = -\frac{\partial \mathcal{F}}{\partial \mu^{(i)}} = \left( \frac{\partial g^{(i)}}{\partial \mu^{(i)}} \right)^T \xi^{(i)} - \xi^{(i+1)}

This compact differential equation explains three long-standing puzzles about the wiring of the mammalian brain. First, it explains why every cortical column contains distinct upper and lower layers of pyramidal cells with opposite projection rules. Deep pyramidal neurons sitting in layers V and VI encode the current expectation at the higher level. Their axons project backward down the hierarchy to deliver top-down predictions, inhibiting expected activity in the lower area just beneath them.

Meanwhile, superficial pyramidal neurons sitting in layers II and III of that lower area compute the residual precision-weighted prediction error. Because the descending prediction has already subtracted out everything familiar, these superficial neurons stay quiet when the world behaves as predicted. Only when reality violates expectation do they fire bursts of gamma spikes forward up the hierarchy to update the higher expectation. At the same time, local error from the next level above restrains that expectation from above.

Second, predictive coding explains a startling anatomical fact about our sensory pathways. In the mammalian visual system, 10 times more nerve fibers run backward from the visual cortex to the thalamus than come up from the eyes. If vision were a bottom-up conveyor belt assembling lines into faces, this massive backward highway would make no sense.

In a predictive brain, the backward highway is the main event. You do not perceive the world by shipping raw camera pixels up to the front of your head. Instead, your brain uses its massive top-down pathways to paint a continuous prediction down onto its early sensory cortices. Bottom-up channels act strictly as an editorial feedback loop, sending up only the news not anticipated by the descending dream. As the neuroscientist Anil Seth puts it, perception is a controlled hallucination reined in by sensory prediction errors.

Third, this framework reveals the biophysical meaning of attention. Not all sensory channels are equally trustworthy at every moment. On a foggy night, visual errors are noisy and low in precision, whereas a snapping twig carries high auditory precision. By releasing neuromodulators such as acetylcholine and dopamine, the brain dynamically adjusts the synaptic precision gain on specific error neurons. Attention is precision optimization. It turns up the synaptic volume on reliable prediction errors while turning down the gain on static and noise, selecting which signals cross into conscious awareness.