Reward prediction error

The difference between what an animal expected to happen and what actually happened.

Positive prediction error occurs when reality exceeds expectation (a better treat than anticipated, an easier task than feared, a friendly response from a new person when threat was expected). Negative prediction error occurs when reality falls short (a smaller treat than anticipated, a harder task than expected, an unfriendly response from someone who had been welcoming).

The brain uses these prediction errors, signalled by dopamine, to update expectations and to drive learning. The mechanism is one of the most important findings in contemporary learning neuroscience and provides a precise computational account of how trial-and-error learning works at the neural level.

The implications for practical training are substantial. Continuous reinforcement, where the same reinforcer arrives every time the behaviour is performed, produces small or no prediction errors once the association is established (the animal expects the reinforcer and gets it; no update needed). Variable reinforcement produces frequent prediction errors (the animal sometimes gets the reinforcer and sometimes does not; each event produces a prediction error that drives further learning). This is one of the reasons variable reinforcement schedules produce such persistent learning.

Surprise rewards (better than expected) engage the reward prediction error system more strongly than predictable ones. This is part of why varying the value of reinforcers (sometimes a piece of dry kibble, sometimes a piece of cheese) can produce stronger learning than always using the same reinforcer. The animal cannot fully predict what will arrive, so each reward provides a small learning signal.

The framework has been demonstrated extensively in rodents, primates, and humans, with similar mechanisms now established in other species including birds. It is one of the more reliably cross-species findings in contemporary behavioural neuroscience.