An AI agent demo has an unfair advantage: it is played on questions its author chose. Production has no such manners. It asks the users' questions, out of order, with dirty data, at two in the morning.
According to the MIT study published in 2025, 95% of enterprise generative AI pilots deliver no measurable return. In the projects we have watched, the cause is rarely the model. It is almost always the absence of three disciplines.
1. Without evaluation, nobody knows whether it works
On demo day, someone tried ten prompts and everything was fine. Three months later, a system prompt was tweaked, a model changed version, a document source moved. Nobody has a number to say whether the system is better or worse than before.
What is needed, and what we build first in a diagnostic: a set of representative cases, drawn from real conversations, each with an explicit success criterion. Fifty well-chosen cases beat a thousand synthetic ones. Then that set runs on every change, and a release can fail on it. It is the only way to evolve an agent without breaking it blindly.
2. Without observability, you discover problems on the invoice
An agent that loops costs a hundred times a simple call. A retrieval chain that returns twenty useless documents doubles the latency. Without traces, these drifts are invisible until month end, when the bill arrives and nobody can name the culprit.
Observability is not one more dashboard. It is the ability, for a given call, to answer three questions: what was sent to the model, what it cost, and why the answer is this one. Traces, latency per step, cost per request: when those three measurements exist, an incident reads in minutes. When they do not, it is guessed at in days.
3. Without handover, the system leaves with its author
The third failure mode is the quietest. The agent works, it is even observed. But it lives in the notebook, or the head, of a single person. They change teams, and the system becomes a box nobody dares to open.
A system in production must be something a team can take over: code in the team's repository, with its conventions and reviews; documentation that says what is fragile and why; handover sessions where the team modifies the system itself, rather than watching someone else do it. We consider an engagement finished when our counterparts no longer need us.
What this changes in the order of things
The natural temptation is to start by building. It is the order that wastes the most.
- Understand what AI can do, what it costs and where it fails, so that decisions are made on facts.
- Measure what exists: read the code that runs, build the evaluation set, cost out what production will require.
- Deliver one use case all the way, with its evaluations, its traces and its handover.
A demo proves an idea is possible. An evaluation proves it is true. A trace proves what it costs. A handover proves it will last. All four are necessary, and only the first is easy.
