Two weeks ago we published the data: 68% of businesses use AI. 77% can't measure it. The response was overwhelming — not because the stat was surprising, but because nobody had a practical answer for "how do I actually start measuring?"
Here's the answer.
Why Measurement Fails (And It's Not Your Fault)
Most measurement attempts fail because they try to measure AI output. How many tokens did the agent generate? How many API calls? How fast was the response?
These are vanity metrics. They tell you nothing about business value.
The real measurement question is: Did the agent improve the outcome of the process it was assigned to?
The Three-Layer Measurement Framework
Layer 1: Process Outcome (Weekly)
For every agent, ask: Did the process improve?
If you deployed an agent to handle customer support tickets, measure:
- Before agent: Average resolution time, CSAT score, escalation rate
- After agent: Same metrics, filtered by "agent-handled" vs "human-handled"
The delta is your agent ROI. Not the number of tickets handled — the improvement in ticket outcomes.
Layer 2: Correctness Verification (Per Action)
This is the layer most businesses skip. For each agent action, can you independently verify it was correct?
- Did the refund agent apply the correct policy?
- Did the pricing agent quote the right tier?
- Did the scheduling agent double-book?
Measurement question: What percentage of agent actions are independently verifiable? Start at 0%. Aim for 100%.
This is where tools like Attest come in — signed receipts that outsiders (auditors, customers, regulators) can verify without access to your internal systems.
Layer 3: Economic Impact (Monthly)
This is the hardest and most valuable layer. It answers: What would have happened without this agent?
- Counterfactual cost: Staff hours saved × hourly rate
- Revenue attribution: Deals closed faster, fewer abandoned carts, higher upsell conversion
- Risk reduction: Compliance violations avoided, errors prevented
A Practical Start
Don't try all three layers at once. Start with Layer 1 for your most active agent this week.
1. Pick one agent. Any agent. 2. Identify the primary business outcome it's supposed to improve. 3. Collect 7 days of before/after data. 4. Write down the delta.
That's your first ROI data point. Next week, add verification (Layer 2). Next month, layer in economic impact.
The 77% of businesses who can't measure AI ROI aren't failing because measurement is hard. They're failing because they're trying to measure AI output instead of business outcomes. Flip the question, and the answer appears.
---
Stop guessing whether your agents are working. Start proving it.