The 50-Point AI Security Checklist for Australian

Most AI security checklists you find online fall into one of two failure modes. Either they are too generic, recycling the same SaaS hardening advice that applies to any cloud product, or they are too theoretical, listing OWASP categories without ever mapping them to a control you could actually implement on a Monday morning.

This article is the seventh in our AI-for-Australian-business series and the first deep dive into security. The earlier instalments covered the governance gap when teams DIY without understanding, the operational reality of running agents in production, the staffing gap most businesses are walking into, the vendor selection questions that surface real risk, ACCC consumer guarantee exposure, and the APRA and ASIC compliance picture for financial services.

AI changes the threat model. You still need ISM-grade hygiene around identity, patching, and logging, but you also need a new layer of AI-specific controls on top. Skipping either layer is what gets your business named in an OAIC notification, an ACCC release, or both.

Part 1: The Threat Model Has Actually Changed

If you are evaluating AI security through a traditional application security lens, you will miss most of what matters. The OWASP Top 10 for Large Language Model Applications 2025 lists ten categories that simply did not exist in classical AppSec.

Here is the short version that every CISO at an Australian business should be able to recite:

The NIST AI Risk Management Framework and its Generative AI Profile (NIST AI 600-1) extend this by adding governance and lifecycle functions. MITRE ATLAS catalogues adversarial techniques against AI systems the same way MITRE ATT&CK catalogues them for classical IT. Treat these three resources as your reading list before signing off on anything.

For the Australian context, two more matter. The Australian Cyber Security Centre Information Security Manual (ISM) is the baseline most regulated buyers expect you to be at. The ASD Essential Eight Maturity Model is the minimum hygiene floor. Neither was written for AI, which is why this checklist exists.

Part 2: The 50-Point Checklist

Rather than a flat list of fifty items, the controls are organised into six families. Each item cites the underlying framework so you can defend it in an audit or board paper.

Family 1: Data Governance and Classification (8 items)

1. Maintain a written data classification scheme (public, internal, confidential, restricted) and require it to be applied before any data set is approved for an AI use case. Maps to ISM control 0271 and ISO/IEC 27001:2022 Annex A 5.12.
2. Complete a Privacy Impact Assessment before any AI system processes personal information. Required posture under OAIC guidance on privacy and AI and APP 1.2.
3. Verify Australian Privacy Principle 6 alignment for every personal information input. The secondary use of data for AI training or inference is the most common APP 6 trip-wire.
4. Apply data minimisation. Send the agent only the fields needed for the task. NIST AI 600-1 GOVERN-1.6 and APP 3.
5. Treat sensitive information (health, financial, biometric, criminal record) as a hard block on public consumer LLMs. Mirrors the OAIC position that organisations should not enter sensitive information into publicly available generative AI tools.
6. Encrypt training, fine-tuning, and RAG corpus data at rest with AES-256 or equivalent. ISM control 0457.
7. Encrypt all data in transit with TLS 1.3, including internal traffic between agents, vector stores, and tool endpoints. ISM control 1139.
8. Document a cryptographic erasure procedure for decommissioned models, embeddings, and adapters. Deleting source data is not enough when a model has memorised it. Addresses LLM02.

Family 2: Identity, Access, and Tool Scope (8 items)

9. Enforce role-based access control across model endpoints, vector stores, agent orchestrators, and observability tooling. ISM control 0445.
10. Require phishing-resistant multi-factor authentication for all admin, developer, and prompt-engineering access. Essential Eight Maturity Level 2 expectation.
11. Issue a dedicated machine identity per agent. No shared service accounts, no personal credentials in pipelines.
12. Scope every tool and API key to the minimum action surface the agent needs. An agent that reads invoices does not get write access to your accounts payable ledger. Direct mitigation for LLM06 (excessive agency).
13. Use just-in-time elevation for retraining, prompt updates, or tool-scope changes, with automatic expiry and approval logging.
14. Maintain an authoritative inventory of every model, agent, prompt, tool, vector store, and integration with an assigned business owner. NIST AI RMF MAP-1.
15. Recertify access and tool scopes quarterly. Access creep on agentic systems is the single most under-monitored risk we see in audits.
16. Apply network segmentation between development, test, and production agent environments. Production data must not flow into non-production environments without anonymisation. ISM control 1182.

Family 3: Prompt and Output Safety (10 items)

17. Validate, normalise, and length-bound every prompt input before it reaches the model. Mitigates LLM01 and LLM10.
18. Treat all third-party content (web pages, PDFs, emails, support tickets) as untrusted. This is the indirect prompt injection case that hit Chevrolet of Watsonville in 2023, where a poorly scoped dealership chatbot was talked into agreeing to sell a vehicle for one dollar.
19. Run a separate guardrail layer for prompt-injection detection. Pattern matching, classifier-based filtering, and known-jailbreak signatures. MITRE ATLAS technique AML.T0051.
20. Define and enforce output policies (no PII echo, no system-prompt disclosure, no executable code in plain channels). LLM02 and LLM07.
21. Treat the system prompt as a secret. Store it in a secrets manager, rotate it on staff change, and never expose it through a tool the agent can read.
22. Sanitise model output before it is rendered, executed, or sent downstream. Output handling is where many real incidents become breaches. DPD's chatbot in January 2024 became a public reputational event because output handling let the bot swear at and criticise the company itself, demonstrating the LLM05 failure mode in a way every customer could see.
23. Apply contextual escaping before model output reaches HTML, SQL, shell, or email channels.
24. Rate-limit per user, per session, and per tenant. Bound token spend at the application layer.
25. Require human-in-the-loop confirmation for any agent action that writes, deletes, sends money, or contacts a customer. Aligns with the human-override pattern in our agent governance article.
26. Red-team every new agent against the OWASP LLM Top 10 and a subset of MITRE ATLAS techniques before go-live, and again after any prompt or tool change.

Family 4: Model, Embedding, and Supply Chain (7 items)

27. Pin model versions in production. A silent provider update can change behaviour overnight. NIST AI RMF MEASURE-2.7.
28. Verify provenance for every base model, fine-tune, adapter, and embedding model. Checksums for open weights, signed attestations for hosted models. Mitigates LLM03.
29. Vet third-party prompt templates and tool definitions the same way you vet third-party code. Prompts are part of your supply chain.
30. Scan vector stores for poisoning indicators. Anomalous chunks, unexpected language, embedded instructions in retrieved content. LLM04 and LLM08.
31. Isolate tenant data in RAG pipelines. Per-tenant index, per-tenant access keys, namespace-level access checks. The classic cross-tenant leak is an LLM08 vector-store misconfiguration, not a model flaw.
32. Test models against adversarial inputs before deployment using a toolkit such as the Adversarial Robustness Toolbox or Microsoft Counterfit. MITRE ATLAS-aligned.
33. Maintain rollback capability for models, prompts, and tool definitions. Treat them all as versioned artefacts with immutable history.

Family 5: Logging, Monitoring, and Incident Response (10 items)

34. Log every prompt, tool call, retrieved document, output, and refusal for high-risk agents. Without this, you cannot reconstruct an incident. ISM control 0580.
35. Centralise AI logs in a SIEM with retention aligned to record-keeping obligations. Five years is the standard ATO retention horizon for business records.
36. Use tamper-evident, append-only log stores for high-risk pipelines. ISM control 0859.
37. Capture token, latency, refusal, and error baselines per agent. Alert on deviation. LLM10 detection.
38. Detect anomalous tool-call patterns (frequency, sequence, target). MITRE ATLAS AML.T0040.
39. Detect prompt-injection signatures and known jailbreak families in inbound traffic.
40. Run a quarterly tabletop exercise covering at least three AI-specific scenarios: prompt injection, excessive agency triggering an irreversible action, and a vector store data leak.
41. Maintain a written AI incident response playbook separate from the generic IR runbook. Standard playbooks do not cover model rollback, prompt invalidation, or tool-scope revocation.
42. Classify AI incidents by severity using a published rubric. Define what triggers a Notifiable Data Breach assessment, which under the OAIC NDB scheme has a 30-day assessment clock from the moment you become aware of a suspected eligible breach.
43. Pre-draft customer, regulator, and media holding statements for the four scenarios above. Speed of response is what limits reputational damage, and you will not draft well at 11pm on a Friday.

Family 6: Vendor and Contractual Controls (7 items)

44. Require SOC 2 Type II or ISO/IEC 27001:2022 certification from every vendor in the AI stack. Request the actual report, not the marketing badge.
45. Require ISO/IEC 27701 (privacy) and ideally ISO/IEC 42001 (AI management system) attestation from any vendor handling personal information at scale. ISO/IEC 42001 is the first ISO standard specifically for AI management systems and is becoming the audit currency for serious AI vendors.
46. For Australian regulated workloads, require IRAP assessment to the appropriate classification level. IRAP is the ASD-endorsed assessment that maps to the ISM.
47. Contractually prohibit the vendor from using your data to train shared models. The Samsung 2023 incident, where employees pasted source code into ChatGPT and the company subsequently banned consumer LLMs, is the canonical reminder of why this clause matters.
48. Require contractual breach notification within timelines that let you meet your own 30-day OAIC clock. Vendor notification of seven days from detection is a reasonable floor.
49. Confirm data residency. Where is data at rest, where is it processed, where is it logged. Sovereign hosting in Australia is a legitimate procurement requirement, not a marketing preference.
50. Require a published list of model providers, sub-processors, and data flows. If a vendor cannot tell you which foundation model is behind their product this quarter, you cannot do your own risk assessment.

Auditor's note: Of these fifty, the ten that disproportionately reduce real risk are items 2, 4, 12, 18, 22, 25, 27, 34, 41, and 47. If you can only fund a first wave, fund those.

Part 3: Mapping to ASD Essential Eight Maturity

The Essential Eight is the right hygiene floor for any Australian business, but it predates the AI threat model. The table below shows where AI-specific controls extend each Essential Eight strategy.

Essential Eight gets you to a defensible IT hygiene baseline. The right-hand column is what makes that baseline credible for AI.

Part 4: Common Security Failures and the Control That Was Missing

Four real, public incidents map cleanly onto the OWASP categories. Each shows what the missing control would have been.

The pattern is consistent. Each of these was a missing operational control the business could have implemented at modest cost, and none of them needed an exotic zero-day.

For more depth on the specific data-leak failure mode, see our analysis of how the wrong AI tools leak business data into training corpora.

Part 5: The 90-Day Implementation Plan

For an Australian business starting from nothing, here is the realistic sequence.

Anything beyond 90 days is continuous improvement. Quarterly access recertification, quarterly red-team, annual third-party assessment for high-risk systems.

A realistic budget for a business with an existing security team looks like this.

That is roughly the cost of one significant Notifiable Data Breach event in legal and incident response fees alone, before any regulator penalty. Under the current Privacy Act maximums, serious or repeated breaches can attract penalties of up to $50 million, three times the benefit obtained, or 30 per cent of adjusted turnover, whichever is greater.

Part 6: In-House or Partner?

Most businesses do not have a dedicated AI security function. The question is whether to build one, augment, or partner. The decision turns on three inputs.

Whichever path you take, governance and accountability stay with you. A partner can deliver controls. They cannot accept the regulator's call for you.

If you want to discuss how this maps to your environment, our AI strategy service and managed AI services pages outline the engagement models. The real-world architectures behind Carbonly and RootCauseAI show how these controls land in production code, not slideware.

Part 7: Eight Questions Every CISO Should Ask Before Sign-Off

Sharp questions for the security exec before any AI system goes into production.

1. Which OWASP LLM Top 10 categories are mitigated by design, and which are accepted residual risk? Show me the control mapping.
2. What is the blast radius of this agent if it is fully compromised? List the systems it can read and write.
3. Where does the data go, in plain English? Provider, region, sub-processors, training use, retention.
4. Show me the last prompt-injection red-team report against this system. Who ran it and what changed afterwards.
5. What is the rollback time for a bad model, prompt, or tool definition? Has it been tested in the last 90 days?
6. What logs would let us reconstruct a worst-case incident, and how long are they retained?
7. Which contractual clauses cover training-data use, breach notification timing, and sub-processor changes?
8. If we received an OAIC notification this morning about this system, what would the first eight hours look like? Who is on the call?

If the team cannot answer all eight quickly and consistently, the system is not ready to ship.

Closing

AI makes specific, named control gaps far more consequential than they were. The fifty items above are the operational floor for an Australian business that does not want to be the next OAIC release or ACCC enforcement story.

If you want a walkthrough against your environment, book a security review with our team. We work with internal security functions, not around them.

Related Reading:

• Why DIY AI Without Understanding Backfires for Australian Businesses - The governance foundation that has to exist before any control checklist is useful.
• Operating AI Agents in Production: The Reality for Australian Businesses - What day-two operations actually look like once the agent is live.
• The AI Agent Staffing Gap for Australian Businesses - Why most teams cannot run this program with current headcount.
• Vendor Selection Questions for Australian AI Buyers - The procurement-stage version of the contractual controls in Family 6.
• AI Agent Governance: Data Access, Privacy, and Human Override - The governance layer that sits above this checklist.

Sources: Research synthesised from OWASP Top 10 for Large Language Model Applications 2025, NIST AI Risk Management Framework and AI 600-1 Generative AI Profile, ASD Information Security Manual and Essential Eight Maturity Model, MITRE ATLAS, ISO/IEC 27001:2022, ISO/IEC 27701, ISO/IEC 42001, OAIC AI guidance (October 2024) and Notifiable Data Breaches scheme, and public reporting on the Samsung 2023, Chevrolet of Watsonville 2023, DPD January 2024, and Replit July 2025 incidents.