Custom AI vs Off-the-Shelf: Decision Framework
The Conversation Nobody Wants to Have
Consider this scenario: a manufacturing company spends $38,000 on an "AI-powered quality control system" from a US vendor. Looked incredible in the demo. Slick interface, real-time dashboards, the works.
Six months later? It's catching maybe 40% of defects. Their old manual process caught 85%.
The vendor's response: "Your data isn't formatted correctly. You need to hire a data engineer."
Translation: They sold a generic model and hoped the problem was generic too.
This pattern happens constantly, and so does the reverse: companies burning $150k building custom AI for a problem a $500/month SaaS tool already handles.
What follows is a practical framework for making the call, built around the five questions that separate a good investment from an expensive lesson.
Question 1: Is Your Problem Generic or Specific?
This sounds obvious until you try to sort your own list.
Generic problems look like:
- Summarising meeting notes
- Basic document OCR
- Email categorisation
- Simple customer FAQ responses
- Code completion and assistance
For these: Buy. Microsoft Copilot, Notion AI, Intercom, Claude, they're trained on millions of examples of these exact tasks. You won't beat them with custom development unless you have very unusual requirements.
Specific problems look like:
- Pricing quotes based on your 8 years of sales data and margin history
- Defect detection for your specific products under your factory lighting conditions
- Compliance checking against Australian regulations plus your internal policies
- RFP responses using your past winning bids and technical specifications
For these: Build. No off-the-shelf tool has your context. The context is the value.
The Litmus Test
Ask yourself: "If my competitor subscribed to the same tool tomorrow, would they have the same capability?"
Build vs Buy: The Quick Decision
Question 2: How Much Is This Problem Actually Costing You?
Businesses often spend 3 months evaluating AI solutions for problems that cost them $8,000 a year.
Before anything else, do the maths:
| Factor | How to Calculate |
|---|---|
| Hours spent on task per week | Ask the team (they'll know) |
| Fully-loaded hourly cost | Salary ÷ 48 weeks ÷ 38 hours × 1.3 (super, leave, overheads) |
| Annual cost | Hours × Hourly cost × 48 weeks |
| Error/rework cost | Estimate based on historical issues |
Example: Accounts payable team manually processing 200 invoices/month.
Invoice Processing Cost Analysis
If the solution costs $50,000 to build plus $500/month to run, you're looking at 18-month payback. That's solid.
If the solution costs $200,000? Walk away.
Question 3: What Happens When It Gets It Wrong?
This is where most evaluations fail.
A chatbot giving a wrong product recommendation is embarrassing.
An AI approving a $50,000 invoice to the wrong supplier is catastrophic.
Risk matrix:
| If AI is wrong... | Impact | Recommendation |
|---|---|---|
| Customer gets slightly wrong info | Low | Automate fully, monitor weekly |
| Internal team wastes 30 mins | Medium | Automate with human spot-checks |
| Financial transaction is wrong | High | Human-in-the-loop required |
| Regulatory/legal breach possible | Critical | Human approval mandatory, AI assists only |
The Australian angle: Under the Privacy Act, you're still liable even if an AI system makes the error. "The algorithm did it" is not a defence. Build your approval workflows accordingly.
Question 4: Do You Have the Data?
This is the killer.
Half of AI projects that fail don't fail because of bad models. They fail because the data is scattered across 14 Excel files, 3 legacy systems, and one guy's email inbox.
Before you build anything, audit:
| Data Requirement | Red Flag | Green Light |
|---|---|---|
| Historical records | "We'd have to export from the old system" | "It's all in our ERP with API access" |
| Consistent formatting | "Every sales rep has their own template" | "We've had standard templates for 3 years" |
| Volume | "We do about 50 of these a year" | "We process 500+ per month" |
| Labels/outcomes | "We don't really track if it worked" | "We have win/loss data on everything" |
Rule of thumb: If you can't pull 12 months of clean, labelled data in a week, you're not ready to build. Start there.
Question 5: What's Your Maintenance Reality?
AI systems decay, which rarely comes up during the demo.
- Models drift as your business changes
- APIs get deprecated
- Regulations update
- Staff leave and take knowledge with them
Off-the-shelf tools: The vendor handles maintenance. You pay monthly. Simple.
Custom builds: You own maintenance. Forever.
Budget reality check:
| Item | Buy (Annual) | Build (Annual) |
|---|---|---|
| License/hosting | $6,000-24,000 | $3,000-12,000 |
| Vendor support | Included | N/A |
| Internal maintenance | ~5 hours/month | ~20 hours/month |
| Model retraining | N/A | Quarterly minimum |
| Integration updates | Usually included | Your problem |
If you don't have someone technical who'll be around in 2 years, think carefully about custom builds.
The Decision Matrix
Putting all five questions together:
Build vs Buy Decision Matrix
| Metric | Before | After |
|---|---|---|
| Generic problem + < $20k annual cost | Low priority | Don't bother (manual is fine) |
| Generic problem + > $20k annual cost | Evaluate options | Buy SaaS |
| Specific problem + < $50k annual cost | Consider customisation | Buy SaaS + customise if possible |
| Specific problem + > $50k annual cost + good data | High opportunity | Build custom |
| Specific problem + > $50k annual cost + bad data | Requires prep work | Fix data first, then build |
| Any problem + critical risk if wrong | High risk | Human-in-the-loop mandatory |
The Hybrid Approach Most People Miss
You don't always have to choose.
Consider a hybrid approach for a law firm scenario:
Situation: A law firm where reviewing a single lease agreement takes six hours Solution:
- Buy: Claude API for general comprehension ($200/month)
- Build: Custom extraction layer trained on their specific clause types ($15k)
- Configure: Human review workflow for flagged items
Total cost: $18,000 build + $200/month running Time saved: 4.5 hours per lease x 20 leases/month = 90 hours/month Annual value: ~$76,500
Payback: 3 months.
The result is a specific tool for a specific problem, with commodity AI doing the general comprehension underneath it.
What We Tell Clients
When someone asks "Should we build or buy?", our answer is usually: "Neither. Not yet."
First:
- Quantify the actual cost of the problem
- Audit whether you have the data
- Define what "wrong" looks like and how often you can tolerate it
- Check if a SaaS tool gets you 70% of the way there
Then decide. This is exactly the kind of structured thinking we apply in our AI strategy consulting engagements.
Most problems don't need custom AI. They need a $99/month tool and a process change.
The ones that genuinely need custom AI are where the competitive advantage sits, provided the build is done properly and maintained afterwards.
Need Help Deciding?
We run a 2-hour Build vs Buy Assessment for $0. We'll audit your top 3 AI opportunities and tell you honestly which ones are worth pursuing and how.
No sales pitch. If a SaaS tool solves your problem, we'll tell you which one. Our AI strategy service helps businesses navigate these decisions with clarity.
Related Reading:
- Build vs Buy AI: The Complete TCO Guide for Australian Businesses - Deeper dive into total cost of ownership calculations
- AI ROI Calculator: How to Justify Your First AI Project in Australia - Quantify the business case for your AI investment
- Contract Review AI: Extract Key Terms and Risks Automatically - Example of a hybrid build approach mentioned in this article
- Legacy System Integration: How to Connect Old Software to Modern AI - Technical considerations for integration decisions
- AI Strategy Services - Our structured approach to AI planning and decision-making
Solve8 helps Australian businesses implement AI that actually works. Based in Brisbane, working nationally. ABN: 84 615 983 732