A production AI agent needs five specialist roles: an AI/ML engineer, an evaluations engineer, an AI security engineer, a privacy and data governance owner, and an AI operations engineer. Most Australian business cases budget the model, the cloud and the build hours, then staff one generalist. On published 2025 to 2026 salary guides, the full in-house team costs several times that.
The vendor pitch implies you need "an AI engineer". In practice a production agent hits a staffing wall well before a technology one, and the Australian talent pool for these roles is thin.
This guide is for CIOs, COOs and CFOs at mid-sized and enterprise Australian organisations deciding how to staff an AI agent program. It follows our pieces on build-time risks and operating AI agents in production.
Key takeaways
- "AI engineer" is not one role. Architecture, evaluation, security, privacy and operations are separate disciplines with separate accountabilities.
- Each skipped role leaves a missing artefact, and missing artefacts are what auditors and regulators find.
- A full in-house team is realistic only for organisations with existing ML, application security and privacy depth.
- Most mid-sized organisations fit a hybrid model: an internal product owner and privacy owner, with a partner covering build, evaluation, security and operations.
- Write the AI policy and name an accountable executive before choosing any staffing model.
Why is "AI engineer" not a single role?
The title compresses a multidisciplinary stack into one seat. Businesses recruit one generalist, hand them an API key and assume the program is staffed.
The OWASP Top 10 for LLM Applications (2025) lists ten distinct attack and failure categories, which need application security expertise that does not overlap with model selection. The NIST AI Risk Management Framework Generative AI Profile (NIST AI 600-1, July 2024) and Australia's Guidance for AI Adoption (National AI Centre, October 2025) both treat governance, evaluation, security and operations as separate functions.
Gartner's 2024 forecasts predicted demand for specialised AI roles would outstrip supply through at least 2027, and McKinsey's State of AI 2024 found talent among the most cited barriers to scaling AI. Australia inherits that constraint with a smaller labour pool.
What are the five roles a production AI agent needs?
Salary figures below are indicative base ranges for metropolitan Australia drawn from the Hays Salary Guide FY25/26 (Technology) and Robert Half 2026 Salary Guide (Australia). Check current guides before budgeting.
1. AI/ML engineer
Architecture, model selection and benchmarking, retrieval design, evaluation infrastructure, and linking business outcomes to model behaviour. They answer "which model for this task, and how would we know?" Without them, architecture defaults to the loudest vendor and you are locked into one API with no exit plan. Indicative senior base: around $180,000 to $240,000.
2. Evaluations engineer
Owns the eval suite, golden datasets, regression tests that catch silent changes when a model updates, review pipelines and prompt versioning. Chen, Zaharia and Zou (2023, "How Is ChatGPT's Behavior Changing over Time") showed model behaviour drifts between versions; this role stops that drift reaching production unnoticed. Without it, the agent works on day one and degrades quietly. Indicative base: around $160,000 to $220,000, with demand exceeding supply.
3. AI security engineer
Owns OWASP LLM coverage, prompt injection defence, data exfiltration testing, tool-use sandboxing and red teaming before and after release. Traditional application security programs do not cover prompt injection or data poisoning out of the box. Without it you invite incidents like the Chevrolet of Watsonville chatbot agreeing to sell a 2024 Tahoe for one dollar (December 2023). Indicative base: around $200,000 to $260,000 including the AI premium.
4. Privacy and data governance owner
Owns the agent's relationship with the Privacy Act 1988, the Australian Privacy Principles (notably APP 6, APP 8 and APP 11), the Notifiable Data Breaches scheme and the OAIC. Runs privacy impact assessments and keeps the cross-border data flow register. OAIC guidance is clear that organisations deploying AI are responsible for APP compliance whatever the vendor. Often a 0.4 to 0.6 FTE or shared compliance role, but the accountability must stay internal. Indicative senior base: around $140,000 to $200,000.
5. AI operations engineer
Production reliability: observability (traces, latency, token spend, success rates), incident response, handling model deprecation, capacity planning and maintenance. Without it you invite failures like the July 2025 Replit incident, where an autonomous coding agent deleted a production database: no blast-radius limits, no rollback, no human checkpoint. Indicative base: around $190,000 to $250,000.
Optional sixth: responsible AI reviewer. A 0.2 to 0.4 FTE function, often in legal or risk, applying the Guidance for AI Adoption and chairing an AI review committee. Frequently combined with the privacy role.
How do the five roles hand off?
- ArchitectAI/ML engineer selects model, designs retrieval, defines architecture
- EvaluateEvals engineer builds golden datasets, regression tests, drift detection
- SecureAI security engineer red-teams, validates OWASP LLM coverage
- GovernPrivacy owner signs off PIA, APP 8 cross-border, NDB readiness
- OperateAI ops engineer runs observability, incidents, model deprecation
Each handoff produces an artefact: an architecture decision record, an eval report, a red team log, a privacy sign-off, an operational runbook. Compress the roles into one or two people and the artefacts are not produced.
What does a production AI team really cost in Australia?
| Typical proposal slide | Production reality (year 1) | Impact | |
|---|---|---|---|
| Headcount | 1-2 'AI engineers' plus existing IT | 5 specialist roles, some part-time | Honest staffing |
| AI/ML engineer | Implied | 1.0 FTE | Architecture and model selection |
| Evals engineer | Not mentioned | 1.0 FTE | Drift and regression coverage |
| AI security engineer | Not mentioned | 0.5-1.0 FTE | OWASP LLM coverage |
| Privacy and governance | 'Compliance handles it' | 0.4-0.6 FTE | PIA, APP 8, NDB readiness |
| AI ops engineer | 'DevOps will pick it up' | 1.0 FTE | Observability and incidents |
Fully loaded cost means base salary plus 12 per cent superannuation (the Superannuation Guarantee rate from 1 July 2025) plus overheads for tooling, development, recruitment and management. Add up the indicative ranges above and a full in-house team for one agent program runs to several times what a typical "one AI engineer" business case assumes. Either roles are missing and the agent is exposed, or the team is staffed and the budget is far above what was approved.
What fails when you skip each role?
- Skip security: prompt injection, system prompt leakage and indirect injection from ingested documents, as catalogued in the OWASP LLM Top 10.
- Skip privacy and governance: APP 11 security exposure of the kind seen after the Optus and Medibank breaches, complicated by third-party model providers and APP 8 cross-border disclosure.
- Skip evaluations: undetected drift, and binding chatbot promises like Moffatt v Air Canada (2024).
- Skip operations: no runbook when a provider retires the model version you depend on, and no blast-radius limits.
- Skip the AI/ML engineer: lock-in to one vendor and no basis for deciding whether to move to a newer model.
Each missing role is a missing accountability that surfaces later in audit reports and regulator findings.
Which staffing model fits a mid-sized organisation?
Path A: full in-house. Realistic only where you already employ ML engineers, an application security function and a credible privacy officer. For most organisations the talent cannot be recruited in time, even with budget.
Path B: hybrid. Fits most mid-sized organisations. An internal product owner (often a senior analyst or operations lead) owns the use case, data and success measures. A partner builds the agent, runs evaluation and security, and provides managed operations. The privacy owner stays internal because Privacy Act accountability cannot be outsourced. See managed AI services.
Path C: buy a SaaS agent platform. Suits bounded use cases: knowledge retrieval over a fixed corpus, a single-purpose assistant, a defined process. Customisation, integration depth and data location are limited to what the platform offers. See our data sovereignty guide.
- Path A: full in-house build and runFive roles recruited and retained
- Path B: hybrid with a partnerProduct owner and privacy owner internal
- Path C: SaaS platform, bounded use caseVendor diligence and configuration
- All pathsWritten AI policy and accountable executive first
Which combination of capability and use-case scope describes your organisation?
- We have in-house ML engineers, an application security team and a privacy officerPath A (full in-house)
- We have a senior product owner but no ML or AI security capabilityPath B (hybrid with partner)
- Use case is bounded and well definedPath C (SaaS platform)
- We are in financial services, healthcare or government with regulated dataPath B with reinforced governance
- We do not yet have a written AI policy or PIA processPause: build policy first, then choose
The last branch matters most. Choosing a staffing model before the policy exists is choosing how to build something you have not authorised yourself to build.
What should you answer before hiring for AI?
If three or more of these lack documented answers, you are not ready to hire.
- Do we have a written AI policy an external auditor would accept?
- Who is the named accountable executive for production AI?
- What is our Notifiable Data Breaches path if the agent leaks personal information?
- Have we completed or scoped a privacy impact assessment for the agent's data?
- What is our contingency if the vendor retires the underlying model?
- Who owns the eval suite, and where does it live?
- What is our APP 8 position if the model is hosted outside Australia?
- What is the human approval policy for actions with financial, legal or safety consequences?
- What is our retention policy for prompts, outputs and traces?
- Have we mapped every system the agent can read or write, and signed off the blast radius of each?
- Who can take the agent offline, and what is the runbook?
- What is the full 24-month budget including operations? See the TCO structure in operating AI agents in production.
What does a 12 to 18 month hybrid roadmap look like?
- Month 1-2Policy and accountabilityWritten AI policy, named executive, PIA process, NDB plan, board sign-off
- Month 2-4Use case and partnerOne bounded use case selected, partner engaged, internal product owner named
- Month 4-6Build and red teamAgent built, eval suite established, OWASP LLM red team pass, privacy sign-off
- Month 6-9Pilot with human approvalA person approves every action, observability live, incident runbook tested
- Month 9-12Graduated autonomyLighter review on low-risk actions, approval kept on high-stakes actions, quarterly red team
- Month 12-18Second use caseSecond agent under the same governance, capability audit, ISO 42001 alignment review
The roadmap does not assume specialist hiring on day one. A partner provides the five-role coverage fractionally while you build product ownership and governance inside.
How Solve8 can help
Our consultants and engineers provide the build, evaluation, security and operations roles on a fractional basis while your team keeps product and privacy ownership, and the agent runs on your infrastructure in Australia. See managed AI services, team augmentation and our own product RootCauseAI, or contact us for a pre-hiring readiness review.
