On September 16, data-governance company Collibra published a survey run by the Harris Poll among 306 US directors and above in data, privacy and AI roles. The headline is that seventy-two per cent say their company's AI initiatives are falling short because of weak data foundations. Two caveats belong next to it: the sponsor sells data-governance software, so it has an interest in that conclusion, and the margin of error is 6.4 percentage points. The figures underneath the headline are more useful than the headline.
Fifty-one per cent of respondents say their staff spend significant hours manually reviewing what autonomous agents produce. Eighty-seven per cent say their teams regularly re-verify that the context the agents work from is accurate. Seventy-six per cent hit critical obstacles moving from pilot to full production in the past year. The report calls the sum of these costs a hallucination tax: oversight, rework and risk that nobody put in the business case.
Where the tax comes from
An agent answers from what it can read. If the price list exists in three versions, if the policy was updated but the old PDF is still on the shared drive, or if two documents contradict each other, the agent will answer with confidence from whichever one it finds, and a person has to catch the mistake afterwards. That is our reading rather than a finding of the survey, but it fits the survey's own diagnosis: the problem sits in the material the agent reads, and the people are the ones absorbing it.
What companies are doing about it
Fifty-three per cent say the reporting line of their AI function has moved closer to the data organization, and eighty-four per cent have defined executive accountability for flawed or harmful agent output. The same week, Fortune reported that chief information officers are racing to put guardrails and central monitoring around agents, and quoted a PwC leader saying that blaming the agent will not hold up as a defence, morally or legally. Accountability is landing on people, so it pays to know which people.
Four ways to shrink the bill
- List what the agent may read, and give each source an owner. One current version of every policy, price list and template; the old ones archived or deleted.
- Measure review time from the first week. The hours per week someone spends checking output are the figure the business case leaves out. If they do not fall over the first few months, the setup needs fixing, not more checking.
- Agree a sampling rule instead of reviewing everything. For example, a person checks a fixed share of outputs each week plus everything above a stated risk level, with an error rate that triggers a rethink.
- Name who answers for the result. One person, in writing, before the agent goes live.
For our part, every agent we deploy begins with an inventory of the sources it is allowed to read and a named supervisor who samples its output, and the review time goes into the monthly report so that this cost is visible from the start rather than discovered at renewal.