Intelligence is predicting what is next: The Elegant Simplicity of LLMs
The Prediction Machine
Our brains are constantly making predictions. When you're reading this sentence, your brain is actively predicting the next word before your eyes even reach it. When you walk down stairs, your motor system is predicting where your foot should land next. When you have a conversation, you're predicting what the other person might say, how they'll react, and planning your response accordingly.
More profoundly, our brains operate in "simulation mode." Neuroscientists have found that our cognitive systems are constantly engaged in "looking into the future" through prediction, preparation, anticipation, and prospection. We're continuously running mental models of possible futures. We map out potential scenarios, evaluate outcomes, and plan responses before events occur. When driving, your brain is simulating multiple trajectories of surrounding vehicles; in conversation, you're modeling the other person's mental state and simulating their possible reactions to what you might say.
Predictive coding theory, which postulates that the brain is constantly generating and updating a "mental model" of the environment that it uses to predict sensory input. The brain builds on anticipation processes where "simulation processes produce predictions" and "every goal-directed action is based on a self-made model of the environment."
This simulation capability lets us navigate extremely complex situations without having to experience them first. We can switch between deep future-oriented simulations and present-moment awareness as needed, focusing our predictive faculties on immediate sensory input when quick reactions are required.
This predictive framework extends beyond the immediate. We predict the weather tomorrow, stock prices next week, and cultural trends next year. The better our predictions and simulations, the more effectively we navigate our world.
Large Language Models (LLMs) operate on a stunningly simple principle: predict the next token in a sequence. That's it. Given all the previous words in a text, what word is most likely to come next?
There's an elegant simplicity here that's worth appreciating. These systems, trained on vast corpus of human-written text (i.e., the internet), have learned to make remarkably accurate predictions about what word should follow another. In doing so, they've (perhaps) inadvertently captured something fundamental about language, knowledge, and intelligence itself.
When an LLM generates text, answers complex questions, or writes code, it's doing this solely through the lens of prediction. It's asking itself: "Given everything I've seen before, what is most likely to come next?"
The Profound Implications
This parallel between human intelligence and token-predicting LLMs raises fascinating questions. For example, is predicting what comes next the foundation for intelligence. Are LLMs all that are needed to create a true general artificial intelligence (AGI) capable of demonstrating advanced human-like behavior?
Revisiting the science, neuroscientists like Karl Friston have proposed that the brain operates as a prediction machine, constantly generating and refining predictions about our sensory inputs and using prediction errors to update our internal models. This is eerily similar to the way LLMs work
The Potential Limits of Prediction
As compelling as prediction is, there are reasons to believe that prediction alone may not be sufficient for achieving true AGI. Obviously, human intelligence involves more than prediction; it includes active exploration, curiosity-driven learning, embodied understanding of physical causality, planning, and intentionality. We don't just predict what will happen; we intervene in the world to make things happen according to our goals.
Our simulation capabilities are particularly crucial here. Cognitive neuroscience research indicates that we don't just predict isolated tokens or events, we simulate rich, multi-dimensional scenarios that integrate sensory, emotional, and abstract concepts. Research on mental simulation shows that our brains engage in neural simulation of action as a unifying mechanism for motor cognition, allowing us to test hypotheses about the world before we act. We can "run the tape forward" on complex situations, toggle between present awareness and future simulation, and dynamically shift our focus based on what matters most in any given moment.
For now, LLMs lack this embodied agency and contextual simulation ability. They can't form their own goals, conduct experiments, or interact with the physical world except through the interface of language. They predict what a human might say next in a conversation about physics, but they don't have an intuitive understanding of physics derived from having a body that falls when it trips. They can simulate text that resembles a human discussing simulations, but they can't actually simulate non-linguistic realities in the rich, multimodal way our brains do.
The Path Forward
There's something profound about the fact that so much apparent intelligence can emerge from the simple task of predicting what comes next. It suggests that prediction is a fundamental component of intelligence, even if it's not the whole story.
Neuroscience has gradually embraced this view, with frameworks like predictive processing becoming increasingly prominent. The theory posits that the brain leverages predictive models when updating beliefs and selecting actions, based on approximate Bayesian inference. According to this perspective, when we encounter errors in our predictions (i.e., events that don't match our expectations) our brains use these “errors" to update our internal models of the world.
Perhaps the path to AGI involves building upon the foundation of prediction that LLMs have established, while adding layers of embodiment, agency, and goal-directed behavior. We might need to incorporate the brain's ability to not just passively predict, but to actively simulate different possibilities and test hypotheses in rich, multimodal environments. Or maybe we'll discover entirely new principles that complement prediction in ways we haven't yet imagined.
Either way, the surprising capabilities of systems built solely to predict the next token should give us pause. They reveal both the power of prediction as a framework for intelligence and the astonishing amount of structure, knowledge, and pattern that exists in human language and thought.
In their elegant simplicity, LLMs have given us a new lens through which to view intelligence itself––not as some mystical essence, but as the everyday magic of accurately predicting what comes next.
For those interested, I have included material for further reading below:
"Solving future problems: default network and executive activity associated with goal-directed mental simulations" by Gerlach et al. investigates how goal-directed simulations activate the brain's default mode network, which is also used during autobiographical memory recall. They found distinct patterns of activation when participants imagined future problem-solving situations.
"Episodic future thinking and episodic counterfactual thinking" by Schacter et al. explores how constructing imagined future events shares neural substrates with remembering past experiences, particularly in the medial temporal lobe.
"Episodic simulation of future events" by Schacter & Addis offers a foundational overview of how humans simulate detailed future scenarios using episodic memory processes.
"Predictions in the brain: Using our past to generate a future" by Moshe Bar emphasizes how the brain draws on past experiences to simulate possible futures, aiding in planning and creative thought.
"Making decisions about the future" by Hoerl & McCormack critiques cognitive biases in simulation-based thinking, suggesting that while it aids decision-making, it can also lead to systematic errors.
"Decision-making, errors, and confidence in the brain" by Rolls & Grabenhorst discusses how the brain encodes confidence levels in predictions, which is central to adaptive decision-making.
