Large Language Models (LLMs) are no longer just next word predictors.

A thoughtful post by Steven Adler on the common refrain that LLMs are just “next word predictors.” This pejorative statement is the dumbest and the most widespread form of cope I see.

I feel torn about whether the RLHF generation should count as ‘just predicting the next word,’ but the latest generation is a very clear “no” for me. This generation is even more different: The AI is presented during training with hard problems, and it thrashes around until it has learned a general skill of problem-solving. I know this sounds pretty wild, but it works; this is the technique11 that lets the AI outperform the top mathematics students, who have also committed themselves to becoming expert problem-solvers.

The latest models are more “path-finder” than “next-word predictor.” At any given moment, these AIs aren’t really trying to predict what word comes next at all; they are selecting the next step to solve a problem, and then executing that problem-solving step; they are pursuing a strong answer in a wide variety of ways, rather than outputting a response directly. (More generally, this type of path-finding AI is called a “reasoning model.”)

Mostly, the AI responses you see ‘in the wild’ aren’t from these models, and so it’s understandable to have out-of-date beliefs about AI’s abilities. Much more common is to see an embarrassing response from, say, Google’s instant search summary, which is free, fast, and rather gaffe-prone. In contrast, ChatGPT 5.2 Pro is excellent quality, but it also costs a ton and takes roughly 15 minutes per query, so almost nobody uses it