Every few weeks an agency founder tells me some version of the same story. The build went fine. The demo went well. The client was excited. And then the project quietly died somewhere between "it works on our test data" and "the client's team actually uses it every day."
The numbers back this up. RAND puts the failure rate of enterprise AI projects at over 80% — roughly double that of ordinary IT projects. MIT's Project NANDA found that about 95% of generative AI pilots deliver no measurable return. And S&P Global found 42% of companies abandoned most of their AI initiatives in 2025, up from 17% the year before. Read those numbers carefully: most of these projects did not fail because the model was bad. They failed at delivery.
Before starting Scopeyard, I delivered AI automation projects across healthcare, recruitment and operations, and before that I ran a product development studio for six years. The AI projects that survived were never the cleverest ones. They were the ones with the most boring, disciplined delivery process.
So here is the checklist I actually use. The thesis is simple:
Successful AI Delivery = Agreed Definition of Done × Evaluated Behaviour × Human Checkpoints × Handover Discipline
Weaken any one factor and the whole product multiplies down to zero.
1. Lock the definition of "done" before you build
"The chatbot answers customer questions" is not a definition of done. It is a wish.
Before you write a line of code or connect a single workflow, write acceptance criteria the client signs. For AI work, that means numbers: task-completion rate, acceptable error rate, latency, cost per task. Practitioners converging on production standards in 2026 suggest useful go-live thresholds of a task-completion rate above 90%, zero safety violations, and cost per successful task inside an agreed budget ceiling.
The right threshold depends on what the system does. A summary being slightly off is annoying. An agent sending a wrong invoice is a liability. Categorise every AI task in the project like this:
| Task type | Example | Acceptable failure rate | Human involvement |
|---|---|---|---|
| Informational | Summaries, drafts, research | 5–10%, with review | Spot-check outputs |
| Transactional | Invoicing, CRM updates, bookings | Below 1% | Approval on high-value actions |
| Safety-critical | Clinical, financial, legal outputs | No autonomous failure | Human verifies every consequential output |
Put this table in the statement of work. It converts "the AI got it wrong" from an argument into a measurement. It also feeds directly into pricing — I covered how in how to price AI automation projects.
2. Map the real workflow, not the described one
Clients describe their process the way they wish it worked. The edge cases live in the gap. In healthcare projects I have seen "we just forward the referral to the specialist" turn out to contain four exception paths, two WhatsApp groups and one person named Susan who fixes everything manually.
During discovery, document: every system the automation touches, every input format that actually arrives (not the clean one in the spec), who overrides what today, and what happens when the process fails now. Gartner predicts that 60% of AI projects lacking AI-ready data will be abandoned through 2026 — and "AI-ready data" is mostly a polite term for "someone bothered to look at the real inputs before building."
3. Build an evaluation set before you build the feature
This is the step most agencies skip, and it is the one that separates AI delivery from normal software delivery. You cannot unit-test a prompt. You evaluate it.
Before the build, collect 50–200 real examples from the client — real emails, real documents, real tickets — with the correct output for each, agreed with the client. This is your golden dataset. Every prompt change, model swap or pipeline tweak gets run against it. When the client asks "did the new version get better?", you answer with a number, not a feeling.
McKinsey's 2026 State of AI work names weak quality measurement as one of the top reasons agent rollouts stall. If you cannot measure quality, you cannot defend your invoice.
4. Put humans in the loop where failure is expensive
Full autonomy is a demo feature, not a delivery strategy. Around 88% of AI pilots never reach production, and one consistent trait of the ones that do is deliberately placed human checkpoints: a review queue for low-confidence outputs, approval gates before money moves or messages go out, and an escalation path when the system is unsure.
Design the confidence routing explicitly. High-confidence outputs flow through; low-confidence ones queue for a named person. Budget for it too — expect roughly 5–15% of cases to need human review in a well-built system, and tell the client that up front. A client who expects 100% autonomy will call a working system broken.
5. Run UAT on the client's live mess, not your clean sandbox
Your test data is polite. The client's data is not. UAT for an AI project means the client's own team runs their real, current work through the system for one to two weeks while the old process still runs in parallel.
Three rules for AI UAT. First, the people who will use it daily do the testing, not the manager who bought it. Second, every failure gets logged against the acceptance criteria from section 1, so "I don't like it" becomes "this is a transactional task and it failed 3% of the time." Third, you fix and re-run the evaluation set before every re-test, so regressions are caught by you and not by the client. A structured review process matters more here than anywhere else in agency work — I wrote about the general version in how to run a better client review process.
6. Instrument monitoring, cost and drift before go-live
AI systems fail silently. An API changes, a model gets upgraded, a prompt that worked in March starts hallucinating in July. If your first drift alert is an angry client email, you shipped an unmonitored system.
Before go-live, the checklist is: logging on every LLM call (input, output, latency, cost), alerts on task-failure rate and cost per task, a dashboard the client can actually read, and a monthly review of sampled outputs against the golden dataset. Watch cost especially — agent-style systems can make 5–20 LLM calls per task, and a model price change or a runaway retry loop shows up on the client's bill, not yours, unless you catch it first.
7. Deliver a handover pack, not a login
The project is not delivered when it works. It is delivered when the client can survive without you being on WhatsApp. The handover pack: an architecture map of every system and credential, versioned prompts with the reasoning behind them, the evaluation set and how to run it, a runbook for the five most likely failures, and a named escalation path.
This protects you as much as the client. When something breaks eight months later because the client's ops team changed a CRM field, the handover pack is the difference between a paid change request and an unpaid blame session.
8. Sell maintenance as part of delivery, not as an upsell
Gartner expects over 40% of agentic AI projects to be cancelled by end-2027, largely on cost and unclear value — and abandoned systems are usually unmaintained systems. Models drift, APIs change, the client's business changes. A monthly retainer covering monitoring, prompt updates, model upgrades and a quarterly eval re-run is not an add-on. It is the last item on the delivery checklist.
Position it that way in the proposal: build fee plus ongoing reliability, priced from day one. Clients accept it easily when it is framed as part of delivery, and resent it when it appears after the first outage.
Final thoughts
None of this is glamorous. There is no clever prompt trick anywhere in this checklist. But the 80% failure rate is not a model problem, it is a delivery problem — and delivery is exactly the thing a good agency is supposed to be selling.
At Scopeyard we built the delivery side of this into the product — milestones, structured client reviews and sign-offs that don't die in email threads — because AI agencies kept telling us their projects stalled at approval, not at build.
The agencies winning AI work in 2026 are not the ones with the best demos. They are the ones whose projects are still running a year later.