Moving an AI pilot to production fails most often because the pilot proved the technology and left the business case open. A proof of value closes that gap: one expensive, repetitive problem, a named sponsor, a baseline measured before the AI runs, real systems, and a go or no-go decision made on numbers within weeks. Run your next AI pilot programme this way.
Key takeaways
- Pilots stall on ownership, baseline, integration, governance and run cost. The model is rarely the problem.
- A proof of value signs its success criteria before anything is built, so the result cannot be argued afterwards.
- It runs on live work, inside your own systems, with a person checking every output until accuracy is proven.
- It ends with a dated go, pause or stop decision, with the monthly run cost already on the page.
Why do AI pilots stall before production?
AI pilots stall because they answer "can it work?" and skip "is it worth running?". Nobody owns the problem, nobody measured the starting point, the pilot never touched real systems, governance was asked last and there is no budget line for running costs.
A proof of value fixes each of those. It tests one production-relevant problem, with an executive sponsor, against a measured baseline, on your real systems.
| Question the executive asks | Open-ended pilot | Proof of value |
|---|---|---|
| Who owns the result? | The IT or innovation team | A named business sponsor who owns the problem |
| What does "worked" mean? | Decided after the demo | Success criteria signed before the build |
| What is it compared against? | Nothing measured | A baseline of today's cost, time and errors |
| Does it touch real systems? | Sample files and a sandbox | Live data, read access to your ERP or document store |
| Has security signed it off? | Usually not yet | Reviewed before any data moves |
| What does it cost to run? | Unknown | Estimated per transaction and per month |
| How does it end? | It drifts | A dated go, pause or stop decision |
If you are a CIO, COO or operations head who has already watched one pilot fade, the sections below answer the questions in the order you are likely to ask them.
How common is the AI pilot to production gap?
The gap is well documented by independent research:
- MIT NANDA, "The GenAI Divide: State of AI in Business 2025" (July 2025): 60 percent of organisations evaluated enterprise-grade AI tools, 20 percent reached pilot stage and 5 percent reached production. The report says most "fail due to brittle workflows, lack of contextual learning, and misalignment with day-to-day operations."
- Gartner (July 2024) predicted at least 30 percent of generative AI projects would be abandoned after proof of concept by the end of 2025, citing poor data quality, inadequate risk controls, escalating costs and unclear business value.
- RAND, "The Root Causes of Failure for Artificial Intelligence Projects" (August 2024) notes that by some estimates more than 80 percent of AI projects fail, twice the rate of IT projects without AI. Misunderstanding of a project's intent and purpose was the most common cause in its interviews.
None of these sources blame the models. MIT found the barriers reported most were change management, lack of executive sponsorship, poor user experience and output quality concerns. Those are design decisions made before anyone writes code.
Want the build detail? The AI proof of concept 4-week framework covers the technical side. This post covers ownership, measurement and how the money gets approved.
Where does an AI pilot programme lose its funding?
A pilot usually loses its reason to exist at one of five points.
No one owns the problem
The pilot belongs to IT or an innovation team. The manager whose team does the work every day was consulted once. When the pilot ends, no budget holder argues for it, because none of them asked for it.
No baseline was measured
The demo shows the AI coding an invoice in seconds. Nobody recorded how long it takes today, how often it goes wrong, or what that costs. With no starting number, there is no improvement to report.
It never touched real systems
The pilot ran on a folder of sample PDFs and a sandbox account. Production means reading from the ERP, writing back to the document system and respecting existing permissions. That integration is usually most of the work, and it was never scoped.
Governance was asked last
Security, privacy and risk see the pilot for the first time when someone asks to go live. Their questions about where data goes and who checks outputs are reasonable. The answers take months and momentum dies.
Nobody budgeted for running it
A working AI system has monthly costs: model usage, hosting, monitoring and an owner. If that line is not in next year's budget, the pilot has nowhere to land. Our guide to operating AI agents after go-live covers that tail. For broader failure patterns, see why AI projects fail in Australia.
How does a proof of value get from pilot to production decision?
A proof of value has a fixed shape. Each step produces something the go or no-go meeting will use.
- Problem nominatedA business owner puts forward one expensive, repetitive problem with a named sponsor
- Baseline measuredToday's volume, handling time, error rate and cost per item are recorded
- Criteria signedSponsor, IT and risk agree the accuracy threshold and go or no-go tests
- Connected to your systemsRead access to real systems and data inside your own environment, in Australia
- Side-by-side runThe AI processes live work alongside the team; nothing reaches a customer
- Results reviewedWeekly comparison against the baseline and the threshold
- Decision filedGo, pause or stop on one page, with run costs and an owner named
The order matters. Criteria are signed before anything is built, so nobody can move the goalposts later.
Who should own an AI proof of value?
Two named people own it, and their jobs differ.
The executive sponsor owns the outcome and the budget. That is usually the COO, CFO or general manager whose cost line carries the problem. They chair the go or no-go meeting and fund production.
The business owner runs the team that does the work today: the AP manager, the service centre lead, the maintenance planner. They supply the baseline, nominate reviewers and decide whether outputs are good enough to use.
IT partners in both roles. It owns architecture, access and security, and it should not own a business problem it does not feel each month.
The Australian Government's Guidance for AI Adoption, published by the National AI Centre on 21 October 2025, puts this first. Its opening practice is "Decide who is accountable", and the implementation guidance asks organisations to "assign, document and clearly communicate who is accountable" for each AI system.
Which problem should go first?
Pick a problem that is expensive, repetitive and measurable. If you cannot count it today, you cannot prove an improvement tomorrow.
Problems that suit a first proof of value:
- Technical document search: engineers, estimators or service staff hunting through drawings, manuals and specifications.
- Maintenance knowledge: planners looking up fault history and procedures across a mine site, fleet depot or building portfolio.
- Operational reporting: weekly packs assembled by hand from three systems that disagree.
- Accounts payable: invoices keyed, coded and matched by hand. See Peppol e-invoicing without rekeying and month-end close automation.
- Customer correspondence: emails and portal requests triaged and routed by hand.
- Quoting: requests read and priced from a price book. Our self-hosted AI quoting agent guide shows one way to keep pricing data onshore.
MIT's research found budgets lean towards sales and marketing, while back-office automation often returns more. A dull back-office queue with a clean count is often the better first candidate.
Which statement best describes the problem you want to test?
- High volume, repetitive, and we can count items and time todayStrong candidate. Start the baseline this month
- Costly, but nobody records volume or handling timeMeasure for two to four weeks first, then decide
- Rare, high-judgement decisions with legal or safety impactKeep a person deciding; test AI as research support only
- The data sits in a system nobody can get access toFix access first; this becomes an integration project
- A board asked for AI but no process has been namedRun a short discovery to find the bottleneck before any build
What baseline makes the business case hold up?
The baseline is the number the sponsor will defend at the budget meeting. Measure it before the AI runs, with the people who do the work.
Record at least:
- volume per week or month
- average handling time per item, from a sample timed by the team
- error or rework rate, and what an error costs downstream
- queue length and wait time, if the work backs up
- the loaded hourly cost of the people doing it
Here is the arithmetic with illustrative numbers. Replace each with your own measured figures.
- Items handled per month (measured)1,200
- Average handling time per item (timed sample)8 minutes
- Team hours per month on this task160 hours
- Items needing rework (measured)6%
- Run cost per item in production (estimated in the proof of value)To be measured
- Go decision rests onHours returned minus run cost
The last line is the point. The sponsor sees the net figure with run cost already deducted. Our AI business case template shows how to present it to a board.
How do you agree success criteria before anything is built?
Write them down, have the sponsor, IT and risk sign them, and do not change them mid-test.
Good criteria cover four things:
- Accuracy threshold: the share of outputs a reviewer accepts without change, set by the business owner.
- Time or cost: the improvement in handling time per item the sponsor needs to fund production.
- Adoption: whether reviewers would choose to keep using it.
- Guardrails: failure types that stop the test outright, such as personal information appearing where it should not.
This matches the Government guidance. Under "Test and monitor", the implementation guidance asks organisations to "define and communicate clear acceptance criteria and test methodologies that reflect the intended use, context and potential risks."
Will it work with our systems and stay in our environment?
It has to, or it proves nothing about production.
A proof of value connects to the systems the work already lives in: the ERP, the document store, the ticketing or maintenance system. It uses read access first and writes back only after the threshold is met. It runs on your infrastructure in Australia or New Zealand, either your cloud account (Azure, AWS or Google Cloud, Australian region) or your own servers, with credentials your team holds.
That answers the security lead's first questions early. Data stays in your environment. Under the major cloud providers' enterprise terms the model provider does not train on it, and your contract should confirm that. Your team can switch it off.
If staff already paste company data into public chat tools, start with a shadow AI inventory and policy. If Microsoft 365 Copilot is on the roadmap, a Copilot readiness permissions audit finds overshared files first.
Will security, risk and governance sign it off?
They will if they are in the room from week one and the design gives them controls they recognise.
The Guidance for AI Adoption sets out six essential practices:
- Decide who is accountable
- Understand impacts and plan accordingly
- Measure and manage risks
- Share essential information
- Test and monitor
- Maintain human control
A proof of value can produce evidence against each: a named owner, an impact note, a risk register entry, an AI register record, test results and a human approval step.
If your organisation is a Commonwealth entity, or supplies one, the Digital Transformation Agency's Policy for the responsible use of AI in government also applies. Its requirements include use case accountability, internal use case registers and impact assessment.
Human oversight settles most debates. During the side-by-side run, AI output goes to a comparison screen, never to a customer or a ledger. Afterwards, a person approves every output before it is used. Our guide to AI agent governance, data access and human override sets out the access model.
How is a proof of value evaluated without risking customers?
It runs alongside your team on the same work they are already doing.
Each week the business owner compares the AI's output with what the team actually did, counting accepted outputs, corrected outputs, misses and time per item. That gives three things a sandbox demo cannot: an accuracy figure from your real work mix, reviewers who watch the tool improve as their corrections feed back, and run cost measured on real volume.
Once the threshold is reached, the AI drafts and a person approves every output before it reaches a customer or system of record.
What does the go or no-go gate look like?
A one-hour meeting with a one-page pack, chaired by the sponsor.
- Weeks 1 to 3Discovery and designProblem confirmed, baseline measured, criteria signed by sponsor, IT and risk, access approved
- Weeks 4 to 9Build and side-by-side runConnected to real systems in your environment; AI runs alongside the team; weekly results
- Week 10Go or no-go gateSponsor reviews results against the signed criteria, run cost and ownership plan
- After a goHuman approval and handoverA person approves every output; code, documentation and support move to your IT team
The pack answers six questions:
- Did the system meet the signed accuracy threshold on live work?
- What is the measured change in handling time and rework against the baseline?
- What will it cost to run each month at full volume?
- Who owns it in production, and who supports it?
- What did security and risk conclude, and what controls remain?
- Which problem could it extend to next?
Go funds production and handover, with the run cost added to next year's budget. Pause names the one blocker, such as data access, and a date to revisit. Stop keeps the baseline, the integration work and the lessons, and moves to the next candidate. A stop after a few weeks is a good result.
What changes when an AI pilot programme runs as a proof of value?
| Open-ended pilot | Proof of value | Impact | |
|---|---|---|---|
| Owner | Innovation or IT team | Named executive sponsor and business owner | Someone argues for funding |
| Starting point | Not measured | Baseline of volume, time, errors and cost | Improvement can be shown |
| Success definition | Agreed after the demo | Signed before the build | No moving goalposts |
| Data and systems | Samples in a sandbox | Live data in your own environment | Proves production fit |
| Governance | Asked at go-live | Reviewed from week one | No late veto |
| Run cost | Unknown | Measured per item and per month | Budget line ready |
| Ending | Drifts | Dated go, pause or stop | A decision on numbers |
MIT's report describes the organisations crossing the divide in similar terms: they judged tools on operational outcomes and sourced AI initiatives from frontline managers, paired with executive accountability.
How do you start a proof of value this month?
- Ask each business unit head to nominate one expensive, repetitive problem and a sponsor willing to own it.
- Pick the candidate with the cleanest count and the easiest data access.
- Start timing the work now, so the baseline exists before any build.
- Book security and risk into the first workshop.
How Solve8 can help
Our consultants and engineers run proof of value engagements inside your own systems, alongside your team, with a dated go or no-go gate and a fixed fee per phase agreed before we start. Our AI strategy service covers choosing the problem, and process automation and system integration cover the build. Talk to us about which problem to test first.
Related reading:
- AI Proof of Concept: A 4-Week Framework
- Running an AI Pilot That Survives Go/No-Go
- Operating AI Agents: The Real Cost After You Build
- What an AI Strategy Consulting Engagement Gives You
Sources: MIT NANDA, "The GenAI Divide: State of AI in Business 2025" (July 2025); Gartner press release (29 July 2024); RAND Corporation, "The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed" (August 2024); National AI Centre, Guidance for AI Adoption (21 October 2025); Digital Transformation Agency, Policy for the responsible use of AI in government.
