Agents stopped being an experiment this year. Industry analysts expect roughly forty per cent of enterprise applications to ship with task-specific agents by the end of 2026, up from under five per cent in 2025. What has not kept pace is the return. Figures compiled from Forrester and BCG this year put about twenty-two per cent of agent deployments at negative return twelve months in, while forty-one per cent reach positive payback inside the same year. Same technology, opposite outcomes.
The interesting part is why. Forrester's root-cause analysis of the deployments that lost money points at three failures, and not one of them is the model: unclear success criteria in forty-one per cent of cases, insufficient tool or data access in thirty-three per cent, and drift in evaluation coverage in twenty-six per cent. Nobody in that list was let down by the quality of the reasoning. They were let down by the setup around it.
Unclear success criteria
This is the big one, and it is the cheapest to fix. If nobody wrote down what «working» means before the pilot started, there is no way to settle the argument twelve months later. «Faster replies» is not a criterion. «Ninety per cent of first-line enquiries resolved without a human, measured on the ticket log, with a satisfaction score that does not drop» is. The discipline is not technical: it is agreeing the number before anyone is emotionally invested in the answer.
Access to the tools and the data
An agent that cannot reach the systems where the work actually lives is a demonstration, not a colleague. A third of the failures come from exactly this: the pilot ran against a copy of the documentation, and then production asked it to read the case management system, the billing platform and the shared drive, none of which it had been given. The integration work is unglamorous and it is most of the project.
Evaluation that stops matching reality
The third failure is the quietest. What you tested at launch stops resembling what the agent does in month six, because the questions people ask change. If the evaluation set is frozen on day one, the numbers keep looking healthy while the service drifts. Alongside this sits a governance gap that the same analyses keep finding: single sign-on never configured, audit logs never connected to monitoring, no isolated environment for the agent to run in. Gartner's parallel reading of the year is blunter still, with the large majority of pilots never reaching production at all.
Three questions worth asking before you sign
- What exactly will we measure, and where is that number recorded today? If the vendor cannot point at the existing log or report, the criterion does not exist yet.
- Which of our systems will it need to read or write, and who is doing that integration? The answer «it connects to anything» is not an answer.
- Who re-tests it in six months, against what, and what happens when the numbers slip? Supervision after launch is a service, not a feature.
For our part, the answer is on the record: every project has a named supervisor, the agent prepares and a person answers for the result, and the maintenance plan includes re-testing and a monthly report rather than a phone number that nobody picks up. That is the part of the work that decides which of the two groups above you end up in.