A proof of concept that cannot fail has proved nothing.
An AI proof of concept is easy to build for showing rather than deciding. It runs on a cleaned sample, behind a demo login, with someone from the build team at the keyboard. It works. Everyone nods. Then the project stalls somewhere between the steering committee and the platform team. Nobody killed it. It never had a way to finish.
Gartner put a number on the pattern in July 2024. It predicted that at least 30 percent of generative AI projects would be abandoned after proof of concept by the end of 2025, "due to poor data quality, inadequate risk controls, escalating costs or unclear business value." In January this year it wrote that the outcome had been worse: at least half abandoned after proof of concept, for the same four reasons. Its June 2025 forecast for agentic AI has over 40 percent of those projects canceled by the end of 2027, and its analyst described most of them as "early stage experiments or proof of concepts that are mostly driven by hype and are often misapplied."
Read the reasons, not the percentages. None of them is "the model was not clever enough". Every one is something a proof of concept could have tested.
A demo, a proof of concept and a pilot are different things
The words get used interchangeably, and that is where the trouble starts.
- •A demo shows that something is possible. Its audience is people who need to imagine it.
- •A proof of concept answers one specific question with evidence, on your data, against a bar agreed in advance. Its audience is the person who has to decide whether to fund the build.
- •A pilot runs the real system on a bounded slice of real work, with real users, and measures it against the current way of doing things.
When a project stalls after its proof of concept, check whether it was really a demo. The fix is not more effort or a better model. It is writing down, before anyone builds anything, which result would make you stop.
Signs you are looking at a demo
If any of these is true, the work in front of you is a demo, whatever the slide calls it:
- •The sample was chosen by the people building the system.
- •Success was defined after the first results came in.
- •Nobody from the team that does the work today has used it.
- •Integration with real systems is listed under "next steps".
- •Nobody can say what one case costs to process.
- •There is no end date, only a next review.
None of these is a failure of effort. They are what happens when a proof of concept starts without a question. Each of them is fixed by writing down, in advance, what the work has to show.
The six things it has to prove
1. The task works on your data, including the ugly cases
Use a sample drawn from real work, not a hand-picked set. Then add the cases your team complains about: the scanned PDF, the supplier who sends invoices as photos, the support ticket that is three questions in one.
A proof of concept that only sees clean inputs proves the model can read clean inputs. You knew that already.
A workable rule: pull the main sample at random from last quarter's volume, then add every known awkward case on top and score the two groups separately.
2. It clears a quality bar that was set before the build
Write the acceptance test first. That means a set of real cases, the answers a process owner agrees are right, and a pass mark the same owner signs. "Looks good" in a review meeting is not a pass mark.
This is not paperwork for its own sake. NIST's AI Risk Management Framework says AI systems "should be tested before their deployment," and that accuracy measurements "should always be paired with clearly defined and realistic test sets" that represent the conditions of expected use. Writing the test before the build is the simplest way to make sure such a test exists, and that nobody adjusts it to fit the results.
Agree what counts as an error, too. A wrong field, a wrong decision and a confident answer where the system should have asked for help are not equally expensive, so they should not be scored as if they were.
3. It can reach the real systems
The gap a proof of concept hides most easily is not quality. It is access.
A proof of concept that reads a CSV export has not proved that the system can authenticate to your ERP, respect its permission model and write back without breaking anything. Prove one real read path and, if the workflow needs it, one real write path into a test environment, using the credentials and scopes it would hold in production.
If that cannot be arranged inside the proof of concept window, that is the finding. It tells you the real project starts with integration, not with prompts.
4. It costs what you think, at the volume you actually have
Measure cost per case on the real sample: model usage, retrieval, any third-party calls, and the minutes a person spends reviewing the output. Multiply by real monthly volume, not by the volume in the demo.
Latency belongs here too. A two-minute answer is fine for a nightly reconciliation and useless in a live chat. Record the slow cases, not just the average.
5. Failure is handled, not hoped away
Every AI system is wrong some of the time. The proof of concept should show what happens then.
- •Does it recognise low confidence and route the case to a person?
- •Can that person see what the system saw and why it decided what it did?
- •Is the action reversible if nobody catches the mistake for a day?
Design the review step now, and put its cost into point 4. A system that is accurate but needs every output checked by hand may not save anything.
6. Someone owns it afterwards
Name a process owner who will run it and an engineering owner who gets the call when it breaks. If neither exists, the proof of concept is a science project, however good its numbers look.
Ownership is also where the business case lives. The process owner is the person who can say whether the result changes how their team works, and that is the measure of value that survives a budget round.
Write the charter on one page
Everything above fits on a page, and the page should exist before the first line of code. A template:
AI PROOF OF CONCEPT CHARTER
Question: The one thing this must answer.
Decision: What we will fund or stop, depending on the answer.
Data: Source systems, random sample rule, list of awkward cases.
Acceptance test: Case set owner, error types, pass mark (signed).
Integration proof: Systems touched, credentials and scopes used.
Cost and latency: Measured per case on the sample, projected to volume.
Failure handling: Confidence threshold, review route, reversibility.
Kill criteria: Results that end the project.
Timebox: Start date, end date, no extensions without a new question.
Owners: Process owner, engineering owner, decision maker.
Decision date: When the decision maker will choose.
The template is deliberately dull. A proof of concept that cannot fill it in is not ready to start.
Decide in advance what kills it
Kill criteria are the most useful line on the page, and the easiest to leave blank.
Choose the three possible outcomes before the work begins:
- Promote. The pass mark was met, the integration path works, the unit cost holds. Fund the build.
- Extend once. One named question is still open, such as access to a second system. Give it a fixed extension with that question written down.
- Stop. The bar was missed, or the cost or the access problem makes the case fail.
Extending more than once is how a proof of concept becomes a permanent pilot. Stopping is a good result: the organisation bought a clear answer for the price of a few weeks, and the charter tells the next team exactly why.
What changes on the way to production
Passing earns the build. It does not shrink it.
Production adds the parts a proof of concept skips on purpose: evaluation on live traffic, monitoring and alerting, access controls, audit logging, rollback, support and a runbook. None of that is optional once real work depends on the system. Our framework for taking AI from prototype to production in six weeks sets those parts out week by week, with the artefacts each one should leave behind.
The charter carries straight over. The acceptance test becomes the first regression suite. The cost model becomes the budget alert. The failure route becomes the review queue. A team that wrote the charter starts the build with a good part of the production specification already written and agreed.
How we run it
The same discipline sits inside our AI discovery sprint: two weeks of mapping a workflow, scoring the options by impact and feasibility, making the build-or-buy call and writing the risk plan and costed roadmap. When the answer is build, the AI development work starts from that evidence rather than from a slide.
Asked to run a quick AI proof of concept and want it to end in a decision? Contact us and we will help you write the charter before anyone writes code.