Taking an AI prototype to production means adding the work AI coding tools rarely do on their own: access control, data protection, behaviour under load, monitoring, and the support and onboarding that let people use it unaided. Tools such as Claude Code, Cursor and GitHub Copilot make the first version cheap. The production work costs about what it always did.
This guide is for two readers. The first is an operations or IT leader whose staff member or contractor built an internal tool, client portal or customer-facing feature with an AI coding tool or a prompt-to-app builder, and who now has people depending on it. The second is a product team holding an AI-built prototype that customers have seen and want. Both face one question: what has to change before the business can stand behind it?
Key takeaways
- AI coding tools speed up scaffolding, boilerplate and tests. Independent research shows they can slow experienced developers on large, familiar codebases.
- Security is the biggest gap: Veracode's 2025 testing found 45% of AI-generated code samples failed security tests.
- Three roadblocks decide whether a prototype survives: security, scale and self-service.
- Decide who owns each part by asking "if this code is wrong, what happens?"
- Most prototypes do not need a full rebuild. Assess part by part: keep, refactor or rebuild.
When do AI coding tools speed things up, and when don't they?
The most rigorous study to date is a randomised controlled trial by METR, published on 10 July 2025. Sixteen experienced open-source developers completed 246 real tasks in repositories they knew well, with and without AI tools. With AI they took 19% longer. Before starting they expected a 24% speed-up, and afterwards they still believed AI had sped them up by 20% (METR, July 2025). The authors caution against reading it as proof that AI fails most developers. It does show that people can feel faster while measurably working slower.
Google's DORA research points the same way at team level. The 2024 report found that a 25% increase in AI adoption was associated with a 1.5% decrease in delivery throughput and a 7.2% reduction in delivery stability (Google Cloud, 23 October 2024). The 2025 report, drawing on nearly 5,000 respondents, found throughput had turned positive but the negative relationship with stability remained. Its summary: AI amplifies what is already there (Google Cloud, 24 September 2025).
Developers themselves are wary. In the 2025 Stack Overflow Developer Survey, 84% of respondents use or plan to use AI tools, yet 46% distrust the accuracy of the output against 33% who trust it, and only 3.1% highly trust it. The top frustration, cited by 66%, is "AI solutions that are almost right, but not quite" (Stack Overflow, 2025).
- METR 2025: task time for experienced developers on their own repos, with AI19% longer
- DORA 2024: delivery stability per 25% rise in AI adoption7.2% lower
- Veracode 2025: AI code samples that failed security tests45%
- Stack Overflow 2025: developers who highly trust AI output3.1%
- Common thread: speed on the first draft, cost on correctnessOwnership decides
Where AI coding tools genuinely help
- Scaffolding a new project, its folder structure, build configuration and a first working screen.
- Boilerplate such as form validation, API client wrappers and data transfer objects.
- Writing unit tests against code that already exists and is understood.
- Mechanical migrations: a library upgrade, a rename across a codebase, a format conversion.
- Internal tools with a small blast radius, where a bad day means an inconvenienced colleague.
Where they slow you down or mislead
- Large, unfamiliar codebases, where the tool lacks the context a long-serving engineer carries.
- Security-sensitive code: authentication, session handling, payment flows, anything touching personal information.
- Ambiguous requirements, where the tool fills gaps with confident guesses instead of asking.
- Long-lived systems, where duplicated code compounds into maintenance cost.
That last point has data behind it. GitClear analysed 211 million changed lines of code from 2020 to 2024 and found the share of lines classed as refactoring fell from 25% in 2021 to under 10% in 2024, while copy-pasted code rose from 8.3% to 12.3% (GitClear, 2025). Duplicated code costs little on the day it is written and keeps costing every time someone has to change it in three places.
For review practices that catch these problems, see our guide to AI code review for software teams.
What changes between an AI prototype and production?
A prototype has to prove the idea works. A production feature has to keep working when the person who built it is on leave, when a customer does something unexpected, and when an auditor asks who could see what. Most AI-built prototypes handle the first and leave the second unaddressed.
| Prototype | Production | |
|---|---|---|
| Users | The builder and a few colleagues | Every customer or staff member, including ones who misuse it |
| Authentication | Shared login or none | Individual accounts, MFA, roles, session expiry |
| Secrets | API keys in code or a .env file in the repo | Secrets manager, rotation, least-privilege keys |
| Data | Sample data, one database | Real personal information, backups, retention and deletion rules |
| Failure handling | Error shows on screen, builder fixes it | Graceful errors, retries, alerts to an on-call owner |
| Monitoring | None | Logs, metrics, traces and cost tracking per feature |
| Change control | Edits straight to the live version | Version control, review, test suite, staged releases |
| Support | Message the person who built it | Documentation, support process, defined response times |
None of the right-hand column is exotic. It is the ordinary craft of software engineering, and it is the part prompt-to-app builders tend to skip because nobody asked for it in the prompt.
What are the three production roadblocks?
Every team that moves software from demo to dependable runs into the same three walls. AI-built prototypes hit them sooner, because the build was fast enough that nobody stopped to plan for them.
Roadblock 1: Security
Veracode tested code from more than 100 large language models across Java, Python, C# and JavaScript. Forty-five per cent of samples failed security tests and introduced OWASP Top 10 vulnerabilities. Java failed 72% of the time, and the models failed to defend against cross-site scripting in 86% of relevant samples. Newer and larger models wrote more functional code, but their security performance stayed flat (Veracode, 30 July 2025).
In practice the security gaps in an AI-built tool cluster in five places:
- Authentication and authorisation. The login screen works, but the API behind it often does not check whether this user may see this record.
- Secrets. Keys for the database, email service or AI provider end up committed to the repository.
- Input handling. Form fields and file uploads pass straight into queries or pages.
- Dependencies. Tools pull in packages freely, sometimes outdated, occasionally ones that do not exist.
- Data residency. Customer data flows to a third-party AI API or a hosting region nobody chose deliberately.
If the feature itself calls a language model, the OWASP Top 10 for LLM Applications 2025 adds its own list, led by prompt injection and sensitive information disclosure.
For Australian organisations this is a legal matter as well as a technical one. Australian Privacy Principle 11 requires reasonable steps to protect personal information from misuse, interference, loss and unauthorised access. Since the Privacy and Other Legislation Amendment Act 2024, APP 11.3 states that those steps include technical and organisational measures (OAIC, APP 11 guidelines).
The OAIC received 532 data breach notifications from January to June 2025. Malicious or criminal attacks caused 59% of them and human error caused 37% (OAIC, 4 November 2025). The ASD Essential Eight controls, including multi-factor authentication, restricting administrative privileges, patching and regular backups, are a sensible baseline. Our AI security checklist goes through them point by point.
Roadblock 2: Scale
A prototype that works for five users can fail at fifty for reasons unrelated to features. Queries that scanned a 200-row table now scan two million rows. Two people edit the same record and one change disappears. A background job that ran in seconds now overlaps with its next run.
AI features add a cost dimension traditional software lacks. Every model call is metered, so a chatty prompt or an unbounded loop can turn a modest monthly bill into an alarming one. OWASP lists unbounded consumption as a top-ten risk for this reason. Scaling well means caching, batching, rate limits per user, and a dashboard that shows cost per feature before finance asks.
Observability is the other half. Without logs, metrics and traces, a production incident becomes guesswork. We cover running costs in operating AI agents in production.
Roadblock 3: Self-service
This is the roadblock teams underestimate most. A tool is self-service when a new user can sign up, get the right access, learn it, get help and pay for it without the builder's involvement. That requires:
- Onboarding that works without a walkthrough.
- Permissions and roles an administrator can manage without a developer.
- Multi-tenancy, so one customer can never see another customer's data.
- A support path with documentation and someone accountable for answers.
- Billing, invoicing and plan limits, if customers pay.
Multi-tenancy deserves particular care. Retrofitting tenant isolation into a single-tenant prototype touches nearly every query in the application, and it is the change most likely to leak data between customers if done in a hurry.
Can a Claude Code prototype go to production without rebuilding it?
Often, yes, in part. No platform takes a Claude Code or Cursor prototype to production by itself. What works is an assessment that decides component by component. The user interface and much of the business logic are frequently worth keeping. The data model, authentication, tenant isolation and deployment pipeline usually need rework by an engineer.
Hosting platforms can run the code, but they do not add access control, tests, monitoring or a support owner. Those come from engineering work on the codebase you already have, which is usually cheaper than starting over.
What should AI build, and what should engineering own?
Most organisations will keep using AI coding tools, so the practical question is where a mistake would land. Low-consequence, easily verified work can go to the tool with a person reviewing the result. High-consequence or hard-to-verify work needs an engineer who owns the design and uses AI as an assistant at most.
If this code is wrong, what happens?
- A colleague is mildly inconvenienced and it is obviousAI builds it, a person reviews before merge
- Wrong numbers reach a report or decisionAI drafts, an engineer writes tests and owns sign-off
- Customer data could be exposed or alteredEngineering designs and owns it, AI assists only
- Money moves, or a regulator could ask about itEngineering owns it with independent review
- Nobody can tell whether it is wrongStop and define acceptance criteria first
Applied to a typical AI-built tool, AI can carry user interface screens, test scaffolding, documentation drafts, data import scripts and internal admin views. Engineering should own the data model, authentication and authorisation, tenant isolation, integrations with core systems such as the ERP, CRM or accounting platform, anything handling payments or personal information, and the deployment pipeline.
The same reasoning sits behind our build vs buy TCO guide: the visible build cost is rarely the cost that decides whether a system is worth having.
What does a typical hardening path look like?
Taking an AI-built prototype to production follows a recognisable sequence. The durations below are a typical shape for a focused tool with one core workflow, not a quote. Payments, many integrations or several customer tenants take longer.
- AssessCode review, threat model, dependency scan
- SecureAuth, secrets, input handling, data residency
- StabiliseData model, tests, migrations, backups
- ObserveLogs, metrics, alerts, cost tracking
- Hand overDocs, support process, named owner
- Week 1AssessmentRead the code, map data flows, list security and scale risks, decide keep, refactor or rebuild for each part
- Weeks 2 to 3Security fixesProper authentication and roles, secrets moved out of code, input validation, dependency updates
- Weeks 3 to 5Stability and scaleTest suite on the critical paths, database indexing, error handling, LLM cost limits
- Weeks 5 to 6OperationsMonitoring and alerting, staged deployment pipeline, backup and restore tested
- Week 6 onwardHandover or managed supportDocumentation, runbooks, support process, and an owner for ongoing changes
Problems that come up repeatedly
- The rebuild that should have been a refactor, and the reverse. The assessment should decide part by part. Often the interface is worth keeping and the data layer is not.
- Nobody owns it after go-live. The person who prompted it into existence has a day job. Software without a named owner degrades.
- The test suite tests the wrong thing. AI-written tests often confirm the code does whatever it currently does. Acceptance criteria have to come from the people who use it.
- Integration surprises. The prototype talked to a spreadsheet. Production has to talk to the finance system, with its own authentication, rate limits and data quality problems. That is system integration work and should be scoped as such.
How do we approach AI-built prototypes?
We build and run software in production, including our own products Carbonly.ai and RootCauseAI. Our team also took a workforce management app used by a company of 1,500+ staff from MVP to live use, and it is still running. Across those builds the pattern holds: the proof of concept is the easy part, and access control, data lineage and support handover decide whether anyone still trusts the system a year later.
We use AI coding tools in our own development and treat their output like a capable junior developer's pull request: reviewed before it ships.
Your next steps this week:
- List every tool in your organisation built with an AI coding tool or app builder, and who relies on each.
- For each one, answer the decision tree question: if it is wrong, what happens?
- For anything touching customer data or money, check where the secrets live and who can log in.
How Solve8 can help
Our custom software development team starts with a one-week assessment and gives you a keep, refactor or rebuild view for each part before anyone proposes a build. Once the tool is live, managed AI services can cover ongoing support. Contact us to arrange a code review.
Related reading
- From Claude Code to production in 30 days
- Build vs buy AI: complete TCO guide
- AI code review for Australian software teams
- Operating AI agents in production
Sources: METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity" (10 July 2025); Google Cloud DORA reports (23 October 2024 and 24 September 2025); Veracode 2025 GenAI Code Security Report (30 July 2025); Stack Overflow 2025 Developer Survey; GitClear AI Copilot Code Quality 2025 research; OWASP Top 10 for LLM Applications 2025; OAIC APP 11 guidelines and Notifiable Data Breach statistics January to June 2025 (4 November 2025); ASD Essential Eight.
