Skip to content
woolly.me
← Articles
ai-readiness series

Why AI pilots stall before production

Most AI pilots don't fail on model quality. They die in the gap between the demo and production, and that gap is infrastructure. What actually stalls them.

Last updated

The demo works. That’s the part that fools everyone. Someone wires a model up to a curated slice of data, it does something impressive in a meeting, and the project gets funded on the strength of a thing that worked once, under conditions production will never reproduce. Then it doesn’t ship, and nobody can quite say why.

The numbers on this are stark and they point the same direction. MIT’s 2025 study found that 95% of generative-AI pilots delivered no measurable P&L impact, and traced the failures to integration and data gaps; the models themselves were mostly fine. Gartner has projected that through 2026, 60% of AI projects will be abandoned for lack of AI-ready data. Deloitte’s State of AI in the Enterprise 2026, a survey of 3,235 senior leaders across 24 countries, found only 42% rate their organization highly on strategic readiness, and operational readiness, the part that actually puts things into production, sits lower still.

People assume the bottleneck is the model. In production it almost never is. The model that impressed everyone in the demo is usually fine. What’s missing is everything between the demo and the deploy.

The gap is the infrastructure, not the intelligence

I run agentic AI against a live platform every day. Parallel sub-agents inventory 50+ microservices across Terraform, ArgoCD, and AWS and drive changes from plan to production on a system serving tens of millions of requests a day. That vantage makes the stall points obvious, because they’re the same ones that stall any change to a production system, only louder.

Data comes first. A model is only as useful as the data it can reach, and in most organizations that data is siloed, stale, and ungoverned, spread across systems that were never meant to talk to each other. The pilot ran on a clean export someone prepared by hand. Production needs the model connected to the real thing, with lineage and access controls, and building that connection is a data-engineering project that dwarfs the model work.

Integration comes next. The pilot mocked the systems it would act on; production doesn’t get to mock them. Now the model has to call real APIs with real rate limits, handle the failure modes those systems actually exhibit, and behave when a dependency is down. This is ordinary platform engineering, and it’s ordinary precisely because it’s the unglamorous work that pilots skip and production demands.

Letting something autonomous touch production

The hardest gap is the last one, and it’s the one I’ve spent the most time building for. Letting an autonomous agent make changes to a live system is a control problem before it’s anything else. The reason most teams can’t move their agent from “suggests things in a sandbox” to “changes production” is that they never built the guardrails that make the second one safe.

What that looks like on my team is concrete. Before any Terraform apply, an adversarial production-safety review runs against the proposed change, and an automated gate blocks anything that’s create-only in intent but references production, all under strict human-in-the-loop review against live cluster state. Every change is machine-verified against dev, staging, and prod accounts before it’s trusted. None of that is model capability. It’s the same infrastructure discipline that lets any risky change reach production safely, applied to a new kind of actor.

This is why I read reliable AI adoption as an infrastructure problem, not a model problem. The point where rollouts succeed or fail is the point where the model stops suggesting and starts acting, and whether you’ve built the platform to let it act safely.

The skill that’s actually short

Deloitte’s survey named insufficient worker skills the top barrier to integrating AI, and I’d sharpen that. The scarce skill isn’t writing prompts, which most engineers pick up in a week. It’s the platform engineering to run agents against real systems at scale without letting them break anything, and that skill is scarce because it’s the same one that’s always been scarce: production judgment, encoded into controls.

An organization that has it ships pilots. One that doesn’t accumulates impressive demos and a growing list of abandoned projects, and blames the models. The next piece in this thread takes the positive case: what an AI-ready platform actually has in it, layer by layer, that a pilot-stalling one doesn’t.

Questions this raises

Why do so many AI pilots fail to reach production?
Rarely because the model is bad. The common blockers are data that's siloed and ungoverned, integration into real production systems that the pilot never touched, and the absence of controls that let something autonomous run safely against live infrastructure. MIT's 2025 study found 95% of generative-AI pilots delivered no measurable P&L, and attributed it to integration and data gaps rather than model capability.
What does 'AI-ready' actually mean for a platform team?
It means the data an agent needs is connected, current, and governed; the systems it acts on are integrated rather than mocked; and there are guardrails that make autonomous change safe, enforced before anything touches production. On my team that's an adversarial production-safety review plus an automated gate that blocks any change referencing prod, all under human review against live cluster state. That's infrastructure work, not model work.
Is the skills gap really the top barrier to AI adoption?
Deloitte's State of AI in the Enterprise 2026 survey of 3,235 senior leaders named insufficient worker skills the number-one barrier to integrating AI. But the missing skill usually isn't prompt-writing. It's the platform engineering to run agents safely against real systems at scale, which is exactly the capability that determines whether a pilot ships or stalls.

Consulting

Dealing with this on your own infrastructure?

I take contract and consulting engagements on exactly this kind of work.

Get in touch