AI Drifts Just Like You Do, and That's the Trap

If your AI keeps surprising you with wrong answers, you don't need a better tool. You need to understand what it actually is. A colour test from 1935 makes it obvious.

Try this. Read the colour these words are printed in: not the words, the colour of the ink. If the word GREEN is printed in red, the answer is "red." Simple. Except your brain hesitates, because part of you desperately wants to read the word instead of naming the colour.

That hesitation is the Stroop effect, documented by a psychologist in 1935 and one of the most reliable findings in all of psychology. Reading is such an ingrained habit that it interferes with the simpler task of naming a colour. It's a humbling little proof that even a sharp mind carries automatic habits it has to fight to override.

Recently, researchers gave that same test to today's leading AI systems and published the results in the journal PNAS Nexus. It's one of the most useful things a manager could read about AI right now, not because the AI failed, but because of how it failed.

It doesn't fail, it drifts, then it falls apart

Give the AI a short list of colour-words and it does fine, a little slower and shakier when the word and the colour clash, much like you.

Then make the list longer. And it comes undone. On the tricky "name the colour, ignore the word" task, accuracy slid from over 90% to single digits as the list grew. In places, worse than blind guessing.

But here's what makes the result so useful. The AI hadn't hit something too hard for it. On those same long lists it could still read every word perfectly, and still name the colours correctly whenever the word and the colour matched. The machinery worked fine. What slipped away was simply the instruction itself ("name the colour, not the word") as the task wore on. The longer the job, the more the model lost the thread of what it had been asked and slid back into its most well-worn habit: just reading.

Humans drift, machines collapse

If that sounds familiar, it should. Anyone who's proofread a long document or checked a hundred rows of a spreadsheet knows the feeling: somewhere around row sixty your attention frays, you miss a rule you were given, you run on autopilot for a line or two. Losing the thread on a long, monotonous task is one of the most human things there is, and watching a machine do it feels instantly relatable. Which is precisely why we extend it the trust we'd give a flagging colleague: we assume that, underneath, it knows the right answer and merely lost focus.

But watch what actually happens as the task gets longer, and the resemblance shatters.

When a person drifts, they drift a little, and then they catch themselves. Accuracy wobbles but holds: on this very test, humans stay around 95% correct even on punishing versions many times longer. And the moment the work gets harder, a person does something the machine never does: they feel the difficulty rising and concentrate harder. Human drift is shallow, self-correcting, and braked by that quiet internal alarm that says focus, this is the tricky bit.

The AI has no such alarm. It doesn't drift and recover; it collapses. The longer the list, the worse it gets, sliding from near-perfect down past the level of random guessing, with no floor and no recovery in sight. It never senses the difficulty climbing. It never grips harder. And it reports every wrong answer in that collapse with exactly the same fluent confidence it used for the right ones. A distracted human is a competent mind having an off moment. This is something else entirely, and that gap, between what it looks like and what it's doing, is the single most expensive misunderstanding in business AI today.

It's a language engine, not a reasoning engine

A tired human still knows what the correct answer is. They could slow down, concentrate, and get it right. The AI was never doing that in the first place.

A large language model does not work out the answer to your question. It predicts likely text: the most probable next words, given everything it has ever read. When you ask it to name a colour, it isn't reasoning about colours; it's producing the response that, statistically, tends to follow a prompt like yours. Most of the time the most probable answer is also the correct one, which is why it feels like reasoning. But it never was. It is an extraordinarily sophisticated engine for "what word probably comes next," and on a long, against-the-grain task, the probabilities drift back toward the obvious, and so does it.

This has one consequence worth pinning to the wall: an LLM gives you a probable answer, not a definite one. Ask it the same question three times and you'll get four different answers. That isn't a fault to be patched. It's what the thing is.

So use it for the right kind of question

Once you see that, the whole "how do we use this safely" problem gets much simpler. Sort your questions into two piles.

Some are probabilistic: there's no single right answer, and "wrong" barely applies. Give me ten subject lines for this campaign. Draft a first version of this email. What angles am I not thinking of for this proposal? For these, an LLM is genuinely brilliant. Variation is a feature, the absence of one correct answer is the whole point, and a fast, fluent, tireless idea-generator is exactly what you want.

Others are deterministic: there is one right answer, and you need the same right answer every single time. How much VAT is owed on these invoices? Which of these contracts expire in Q3? Reconcile this ledger. For these, asking an LLM directly is asking for trouble. It will hand you a confident, plausible answer that's right four times and quietly wrong the fifth, and it won't tell you which was which.

So don't ask the LLM the question. Ask it to write the code.

This is the move almost no one makes, and it changes everything. There are many ways to write a small piece of code that gets the right answer, and unlike the model's off-the-cuff reply, code does the same thing every time you run it. It's a recipe, not a guess. Better still, you can read it, test it, and debug it; you can hand it to someone and have the logic checked. The model is superb at writing that code even when it's unreliable at doing the sum in its head. Let it build you a repeatable tool instead of performing a one-off magic trick.

You don't have to be technical to ask for this. "Don't just give me the answer; write me a script that produces it, and explain what it does" is a sentence any manager can use.

A few smaller habits help too: break big jobs into small batches, because length is what breaks it; check the end of long outputs, because that's where the drift collects; and treat any instruction that fights the obvious (ignore this, flag only that, do the opposite of the usual) as the fragile, high-risk request it really is.

The takeaway

People love to call a large language model "a glorified autocomplete," usually as an insult. They've got the description exactly right and the tone exactly wrong. Autocomplete this good is a small miracle, and naming it honestly tells you precisely what to reach for it for, and what not to.

Want inspiration, a first draft, a dozen ideas, a sounding board, an answer where nothing is truly wrong? It's the best tool you've ever had. Want the same correct number every time? Don't ask the autocomplete. Ask the autocomplete to write you something that can.

The teams that win with AI won't be the ones with the cleverest tools. They'll be the ones who stopped being surprised, because they finally understood what the tool was doing all along.