perspective 18 Aug 2026 · 2 min read

When Saying Is Not Doing

Someone spent nine turns trying to identify a medication from a description. Three times the assistant really did search. Six times it said it was searching and did nothing. Intermittent, which is the worst kind: it does…

ToRun Team ToRun Team

Someone spent nine turns trying to identify a medication from a description. Three times the assistant really did search. Six times it said it was searching and did nothing. Intermittent, which is the worst kind: it does not read as a defect, it reads as being lied to.

Our first instinct was the prompt. It usually is. And it was wrong.

The tool was there the whole time

Web search shipped in full on every one of those calls, and the same model on the same prompt did call it in the same conversation. A tool that is being withheld cannot fail intermittently.

What had changed was the thinking budget. Letting a family of models into Auto meant flooring their reasoning effort to keep them affordable — and on the path those models take, that setting was applied whether or not anyone asked for it. Floored that far, the model cannot make a tool call at all. It can only talk about making one.

The same shape, twice more

Once you have the pattern, you start seeing it. Ask for fifty rising coins with an analysis and the spreadsheet arrives with six rows: a real, openable file that looks finished until you read it. The instruction the model worked from described the shape of the file and said nothing about scale, so a representative sample satisfied it.

And after making a document, the spoken assistant read the entire download address out loud, character by character, for ninety seconds. It was not ignoring its instructions — one instruction said never read addresses aloud, another said show the link. On a spoken surface, showing means saying.

The rule we keep relearning

An assistant that narrates an action it cannot perform is worse than one that says it cannot. All three of these are fixed at the layer that made the false statement possible, not by asking more firmly.