The consultant you have been briefing#
A consultant answers in a written recommendation. You brief them, they take time to think, and they come back with reasoning you read and weigh. The words are the deliverable. That is the engagement, and it is worth what it costs.
An instrument answers in a reading. A thermometer, a pressure gauge, a smoke detector. One value, plus how far to trust it, printed on the side. Nobody reads a thermometer for its reasoning.
Both are sound ways to get an answer. They are not the same way, and they are not priced the same.
Since 2022 there has been one option on the menu, and it was the consultant. Every AI initiative — the budget approved, the pilot run, the vendor chosen — was built around briefing something in words and getting words back. That worked. It changed what companies could attempt, and the excitement was earned. It also quietly set the shape of every system built on top of it.
The difference that matters later is where the confidence sits. A consultant’s confidence lives in the prose. An instrument’s lives in the tolerance printed on the side. Only one of those two can be read by software.
What we did with the consultant’s answers#
Here is the part worth sitting with.
For four years we took a thing built for deliberation — slow, one step at a time, wordy, speaking to a person — and used it to make the millions of small judgments that live inside software. Those judgments were never deliberations. Is this refundable. Which queue does this go to. Does a person need to see this. They are reflexes: fast, bounded, wordless, and useful only if they carry a sense of how sure they are. We wrapped each one in a paragraph of prose, paid seconds and cents for the privilege, and then wrote code to throw the paragraph away. The mistake was never capability. It was running deliberation on reflexes, and mistaking the narration for understanding.
That last sentence is the whole argument. The model was never too weak. It was the wrong shape for the job — and because the paragraphs read so well, almost nobody noticed. We put a consultant behind every decision in the business, including the forty thousand a day that only ever needed a gauge.
The tell
In a great many production systems the model writes a paragraph and a few lines of code keep four characters of it. You paid for the paragraph. You shipped the four characters.
Two kinds of question#
Split the work you actually do into two piles.
An essay question is open. The answer is not on any list. Draft this, summarise that, research this market, write this code, explain this to a customer. A person reads the output, and the words themselves are the point.
A reflex question is closed. There are six possible answers, or two, or forty. Route it, classify it, score it, screen it, decide whether a human should look at it. Nobody reads the answer — code acts on it immediately.
Almost every organisation has been answering reflex questions by briefing a consultant, for the simple reason that the consultant was the only thing on the menu.
So what is a System One model?#
The name comes from Daniel Kahneman: System One is fast, automatic thinking; System Two is slow, effortful deliberation. A System One model is built to do the first kind.
Three things make it different, and none of them is about being cleverer.
- You declare the answers first. Before you call it, you state what it is allowed to return — one of these six queues, yes or no, a score from one to five. It cannot return anything else.
- It tells you how sure it is. Every answer arrives with a number. Your code sets the rule: above this line, act; below it, ask a person. That rule is yours, written down, and reviewable.
- It is fast, and it runs in parallel. There is no sentence to compose, so the answer comes back in tens of milliseconds — and you can ask a hundred independent questions at once instead of in a chain.
The same decision, minus the paragraph. Illustrative.
What it is not#
Four honest limits. It is worth saying them yourself, before a vendor has the chance not to.
- It does not guarantee a right answer. It guarantees a valid one. Choosing from your list is not the same as choosing correctly. Structure is not truth.
- It cannot name an answer that is not on the list. If a case really belongs in a seventh category you never wrote down, it goes confidently into one of your six. The gap moved. It did not close.
- It does not write. Anything a human being reads still needs the consultant.
- The confidence number can drift. It is a promise about the kind of data the model was trained and tested on. When your data changes — new product, new region, new season — the number can stay high while the accuracy quietly falls. It will not announce that.
One more: the speed and price figures circulating right now are published by the companies selling the product, and nobody independent has reproduced them. Treat the shape as real and the benchmarks as unverified, and you will not be embarrassed either way.
Where each one fits#
Drafting, summarising, explaining, replying to a person
Consultant. The output is words for a human, which is exactly what this shape was built for.
Research and synthesis across messy, contradictory sources
Consultant. The answer is not on any list, and reconciling a mess is a genuine strength.
Coding and multi‑step planning where step three depends on step two
Consultant. Real deliberation. Leave it there.
Routing, classifying, tagging, triage
Instrument. A short list, high volume, and no reader at the other end.
“Is this complete enough to approve automatically?”
Instrument. Yes or no plus a confidence number is the entire answer, and the threshold is a policy you own.
Screening every payload for personal data before it goes anywhere
Instrument. It runs on every single call, so it has to be cheap and fast or it gets switched off.
Checking the consultant’s output before it reaches a customer
Instrument. A second, independent judgement on the first one’s work.
Deciding whether a human should look at this at all
Instrument. This is the escalation decision itself. It should never be buried inside a paragraph.
In practice the answer is usually both. An instrument in front — screen the input, route it, decide whether it is allowed. The consultant in the middle, doing the work that genuinely needs words. An instrument behind it, checking the output before it ships. Most of the systems worth building over the next two years look like that sandwich, not like a choice between two vendors.
Security, grounding, quality — what actually changes#
These were the three reasons initiatives stalled over the last four years. Here is what the new shape moves, and what it leaves exactly where it was.
Security
What leaves your walls stops being a mystery. A consultant is briefed at run time with whatever the query happened to return, and nobody reads that brief. A declared schema means you can list the fields that go out, in advance, on paper. That is real. It is not a control: a field called notes can still carry a national ID, and the model still runs somewhere. Where it runs is still your question to ask.
Grounding
Unchanged, and worth saying plainly. An instrument decides; it does not fetch. If retrieval hands it the wrong document it will pick the wrong queue — faster and cheaper than before. What you do gain is somewhere to put the doubt: low confidence becomes the signal to go and get more context before deciding. That branch did not exist previously.
Quality
The biggest change of the three. Today quality is estimated in aggregate, offline, by an evaluation suite — “94% on our test set.” That can never tell you whether this answer is in the 6%. A number on every decision moves quality from a quarterly measurement to a per‑call property, and from an engineering exercise to a threshold a named person signs.
If that last point sounds familiar, it is the argument in Confidence is the new bottleneck arriving from the other direction. There it was people who were badly calibrated about AI. Here it is the model, finally able to say so itself.
What survives an audit
Whichever shape a decision uses, the thing that survives an audit is the record: what went in, what came back, how sure it was, and what your policy did about it. That is what an Inspector keeps in Watch mode — before anything is allowed to run.
The skills question changes shape#
One of the honest blockers of the last four years was people. Not enough engineers who could take a promising demo and turn it into something an enterprise could depend on. That problem does not vanish, but the job changes, and mostly in your favour.
Writing a good list of allowed answers is not machine learning. It is the work your best analysts and operators already do — knowing that a refund request has six real outcomes rather than four, and that the seventh turns up every November. Setting a threshold is a risk decision, which is what your risk people are for.
So the scarce skill moves away from prompt craft, which was never really a discipline, and towards two things you already employ: people who know the domain well enough to enumerate it, and people who can say what level of doubt is acceptable. That is a considerably easier hiring problem than the one you had in 2023.
From one model to a portfolio#
One vendor, one endpoint, every use case. Simple to buy — and you are paying frontier prices for work a lookup table could do. Quality is whatever the vendor shipped this month.
A cheap model for easy work, a large one for hard work, perhaps something tuned on your own data. Better economics, and a real improvement. Still one shape of answer everywhere: words in, words out.
Essay questions to the consultant. Reflex questions to an instrument, with your list of answers and your threshold. The tailoring that matters is not the model — it is your taxonomy and your tolerance for doubt.
This is the step that turns “we use AI” into something specific to your industry. A hospital, an insurer and a manufacturer all buy roughly the same models. What differs is the list of allowed answers, who wrote it, and what happens at 0.85. That is your advantage, and it is not for sale.
Five questions for your next review#
“How many decisions in our systems are a choice from a short list?”
Nobody has ever been asked for this number. Finding it takes about a week, and it changes the conversation, because it sizes the opportunity in units you can actually act on.
“Who wrote the list of allowed answers, and when was it last reviewed?”
Every classification you run already has a list. It was usually written once, by one person, in a hurry, for a pilot. It is now a policy document with no owner.
“What happens today when the model is unsure?”
If the answer is “nothing”, there is no escalation path — there is a hope. This is the cheapest gap to close and the one that shows up in incidents.
“How much of our AI spend is paying for words nobody reads?”
Look at the steps where the output goes straight into code rather than to a person. That is the budget that can move, and it is usually larger than anyone expects.
“If one of these decisions is wrong, who notices, and when?”
Name the person and the moment. If you cannot, the review path does not exist yet — whichever shape of model you are using.
What this does not tell you#
This category is weeks old in public. The shape of the idea looks clear and, I think, durable. Which vendors survive it is not clear at all, and the performance figures in circulation come from the companies selling the product, with no independent reproduction so far.
So the useful move is not to procure. It is to find out how many of your decisions are reflexes, and who owns the lists. That work pays off whichever product you eventually buy, and it is worth doing whether or not this particular category lasts.
None of it retires the consultant. Open questions still need one, and the best work of the last four years came from finally having one on call. What changed is that there are now two shapes instead of one — and the interesting question in your next review is no longer which model, but which kind of question are we actually asking.
Related
