An AI readiness assessment checks whether an organisation can run a specific AI use case before any pilot is approved. In Australia it should score six dimensions: data availability, data quality, data governance, organisational capability, compliance posture (Privacy Act, sector rules) and executive alignment. The output is a measured decision: proceed, remediate or stop.
Look back through post-mortems of failed AI programs and the same finding appears. The pilot did not fail because the model was wrong. The organisation was unready before the pilot was approved: data in the wrong shape and place, unclear ownership, the person who knew the data long gone, no accountable sponsor, no written stop rules.
MIT's State of AI in Business 2025 research found most enterprise generative AI initiatives produced no measurable profit-and-loss impact. Gartner has estimated that poor data quality costs organisations an average of US$12.9 million a year. The work that decides pilot success happens before the pilot.
This guide is for CIOs, CDOs, operations managers and CFOs at mid-sized and enterprise Australian organisations. It sits before our AI pilot project success factors playbook: this is the gate the pilot passes through. For a quick self-check, try our online AI readiness assessment.
Key takeaways
- AI readiness is mostly a data and organisation question, scored against one specific use case.
- Use DAMA-DMBOK and ISO/IEC 25012 to define data quality in measurable terms.
- Add an Australian layer: Privacy Act APPs, the Notifiable Data Breaches scheme, APRA CPS 230 and 234, and sector rules.
- Score 30 items from 0 to 3. Any zero is a blocker regardless of total.
- A fixed four-to-six-week assessment ends in a documented decision, not a narrative.
What are the six dimensions of AI readiness?
Each dimension must clear a defined threshold before a pilot is approved.
- Data availabilityDoes the data exist, is it accessible
- Data qualityAccuracy, completeness, timeliness
- Data governanceOwnership, access, audit trail
- Org capabilityWho builds and operates it
- Compliance posturePrivacy, NDB, industry rules
- Executive alignmentSponsor, KPI, stop rule
The order matters. You cannot assess quality on data you cannot reach, govern data that is not catalogued, staff capability you have not scoped, or assign sponsorship to a problem nobody on the leadership team owns.
Organisational capability is covered in our AI agent staffing gap analysis. Compliance overlaps with consumer law and AI, APRA and ASIC compliance and the 50-point AI security checklist. This guide does most of its work on data quality and governance.
How do you measure data quality for AI readiness?
Teams argue about "good data" because they have not agreed a definition. Two references settle it. DAMA-DMBOK 2 defines data quality through measurable dimensions. ISO/IEC 25012:2008 overlaps strongly and sits alongside the ISO 8000 data quality series.
| Definition and measurement | Common Australian failure pattern | |
|---|---|---|
| Accuracy | Data matches reality; measured by sampling against ground truth | Customer addresses with invalid postcodes; ABN fields holding free text |
| Completeness | Required fields populated; measured as % non-null on critical attributes | Time-series gaps where systems were offline; optional fields the use case treats as mandatory |
| Consistency | Same entity represented the same way across systems | One customer in Xero, MYOB, CRM and ERP with four IDs and three name spellings |
| Uniqueness | No duplicate records; measured by entity-resolution match rates | Acquisitions doubled customer records; never reconciled |
| Timeliness | Data current enough for the decision being made | Inventory updated nightly but used for real-time pricing |
| Validity | Conforms to format, range and business rules | Date fields holding strings; postcodes outside Australia; negative prices |
| Accessibility | Reachable by the systems and people that need it | Data locked in a legacy system with no API; one person knows the export |
| Integrity | Relationships and constraints preserved | Orphan invoice lines; foreign keys broken during migrations |
The dimensions are not equal. A generative AI use case can tolerate weaker completeness if people verify outputs, but not weak consistency, because the model retrieves contradictory facts and answers confidently and wrongly. A classification model cannot tolerate weak validity. An agent workflow cannot tolerate weak accessibility. Weight each dimension against the use case in front of you.
How does a pre-pilot data readiness assessment run?
It is a structured four-to-six-week discovery ending in a documented decision. No platform purchase substitutes for it. DAMA-DMBOK covers the activities, and the EDM Council's Cloud Data Management Capabilities framework covers the cloud governance overlay.
- Week 1Data source catalogueIdentify every system holding data relevant to the use case. Capture owner, technology, volume, refresh cadence, access path.
- Week 2-3Lineage and ownership mappingFor each critical data element, document where it originates, what transforms it, and who is accountable for quality.
- Week 3-4Data quality scoringSample and measure the eight dimensions against the use case requirements. Produce a quality heat map.
- Week 4-5Gap analysis vs use caseCompare the scorecard with use case requirements. Separate hard blockers from soft constraints.
- Week 5-6Remediation plan or halt decisionEither a remediation plan with timeline, or a documented decision to defer.
The output is three documents: a data inventory, a quality scorecard and a gap analysis. A small team produces them, typically a data analyst, a process expert and an architect walking the data with its owners.
Treat it as measurement, not documentation. If the scorecard has no numbers, nothing was measured. Sampling 100 records and counting violations per dimension beats a 20-page narrative about how the data feels.
What Australian obligations does an AI readiness assessment need to cover?
Generic frameworks are written for global audiences. Add an Australian layer.
| Obligation | What the assessment must verify | |
|---|---|---|
| Privacy Act APP 1 | Open and transparent management of personal information | Privacy management plan covers the AI use case |
| Privacy Act APP 10 | Quality of personal information | Personal information used by AI is accurate, current, complete |
| Privacy Act APP 11 | Security of personal information | Encryption, access control, retention and deletion defined |
| Privacy Act APP 8 | Cross-border disclosure | Vendor processing locations identified, contractual protections in place |
| Notifiable Data Breaches scheme | Mandatory breach notification | Breach response plan covers AI vendor incidents |
| APRA CPS 230 and CPS 234 (financial services) | Operational risk and information security | AI vendor in the operational risk register and material service provider list |
| AASB S2 climate disclosure | Climate-related financial disclosures | Emissions and energy data quality sufficient for disclosure |
| Modern Slavery Act 2018 | Annual supply chain statement | AI vendor due diligence captured in reporting |
| Guidance for AI Adoption (National AI Centre, 2025) | Six essential practices for responsible AI | Data governance and accountability practices demonstrably met |
| Fair Work and state surveillance laws | Workforce monitoring and decision constraints | Workforce data used by AI complies with surveillance and decision rules |
For detail, see our Privacy Act vs GDPR comparison and Fair Work compliance guide. Compliance posture is a yes-or-no question: either the use case has been mapped and gaps fixed, or it has not.
What is in the 30-point AI readiness checklist?
Thirty questions in five sections of six. Score each 0 to 3: 0 not started, 1 documented intent, 2 partially in place, 3 fully in place and evidenced. Maximum 90.
Section A: Data foundation
| Question | Evidence of 3 / fully in place | |
|---|---|---|
| A1 Inventory | Is there a documented inventory of every data source relevant to the use case? | Catalogue with owner, technology, volume, refresh cadence, access path for each source |
| A2 Lineage | Is the lineage of critical data elements documented end to end? | Origin system, transforms, destinations and accountable owner per critical element |
| A3 Quality scoring | Has data been scored against ISO 25012 dimensions for the use case? | Quality scorecard with measured percentages, not narratives |
| A4 Access | Can the AI system access the data through a supported integration path? | API or warehouse contract documented; no manual exports |
| A5 Retention | Are retention and deletion rules defined and enforceable? | Policy documented; technical enforcement in place |
| A6 Residency | Are data residency requirements known and met? | Storage and processing locations documented; vendor contracts aligned |
Section B: Use case definition
| Question | Evidence of 3 / fully in place | |
|---|---|---|
| B1 Bounded scope | Is the use case bounded to one process, one team, one population of data? | Scope statement signed by sponsor; out-of-scope items listed |
| B2 Measurable outcome | Is there a quantitative success metric tied to a business outcome? | KPI with target, measurement method and reporting cadence |
| B3 Baseline | Is the current state baselined (cost, time, error rate)? | Baseline measured in the four weeks before pilot start |
| B4 Stop rules | Are stop criteria written down and signed by the sponsor? | Specific conditions under which the pilot is stopped, with timing |
| B5 Executive sponsor | Is there a named accountable executive sponsor? | Sponsor named in the business case with decision rights |
| B6 Risk class | Is the use case classified for regulatory and ethical risk? | Risk classification against the Guidance for AI Adoption and applicable law |
Section C: Organisational readiness
| Question | Evidence of 3 / fully in place | |
|---|---|---|
| C1 Roles | Are the build, run and oversight roles defined and resourced? | Named individuals or vendor contract for each role through the pilot |
| C2 Training plan | Is there a training plan for users, operators and reviewers? | Training content, schedule and competency check defined |
| C3 Change management | Is there a change plan for affected staff? | Communication plan, feedback mechanism, escalation path |
| C4 On-call | Is operational coverage defined for when the AI misbehaves? | On-call roster, escalation paths, vendor SLAs documented |
| C5 Internal communication | Are internal stakeholders informed and aligned? | Stakeholder map, communication schedule, FAQ |
| C6 Success criteria | Are success and failure criteria shared across the organisation? | Decision criteria published before the pilot starts |
Our operating AI agents in production and AI change management guides go deeper on this section.
Section D: Compliance posture
| Question | Evidence of 3 / fully in place | |
|---|---|---|
| D1 Privacy Impact Assessment | Is a PIA completed for the use case? | PIA completed per OAIC guidance, signed and filed |
| D2 Security review | Has the vendor or build been security reviewed? | Threat model, control mapping, penetration test results |
| D3 Board sign-off | Has the board or executive committee approved the use case? | Minuted approval with risk register entry |
| D4 Contract review | Have vendor contracts been reviewed for data, liability, exit? | Legal review complete; DPA, SLA, exit clauses verified |
| D5 Insurance review | Have cyber and PI policies been reviewed for AI coverage? | Broker confirmation that the use case is covered |
| D6 ACL exposure | Has consumer-facing exposure been assessed under Australian Consumer Law? | Misleading-conduct risk reviewed for AI outputs to consumers |
Section E: Technical posture
| Question | Evidence of 3 / fully in place | |
|---|---|---|
| E1 Hosting | Is the hosting model decided and documented? | Region, environment and network topology specified |
| E2 Integration | Are integrations with source systems specified and tested? | API contracts, error handling, retry behaviour documented |
| E3 Observability | Are logs, metrics, traces and evaluation hooks in place? | Dashboards live; alerts routed; sample-and-review process defined |
| E4 Evaluations | Is there an evaluation plan with regression sets? | Eval suite for accuracy, safety and edge cases; runs on every change |
| E5 Exit plan | Can you exit the vendor or solution within a defined window? | Exit clause, data extraction procedure, fallback process |
| E6 Audit logs | Are AI inputs, outputs and decisions logged for audit? | Immutable audit log with retention aligned to regulatory needs |
See AI vendor selection questions and AWS, Azure and Google Cloud AI services in Australia for this section.
How do you read the AI readiness scorecard?
Total the 30 scores and match the band. Score honestly: inflating it only moves the cost later.
- 75-90: Ready, proceed to pilotDocument residual risks; start
- 50-74: Targeted remediation neededAddress 0 and 1 scores first
- 30-49: Foundational work requiredPause for 3 to 6 months
- 0-29: Not ready, defer AI programBuild foundations first
A high total with any zero is not ready: zeros are blockers. And the score belongs to a use case, not the organisation. The same business may score 85 for an internal knowledge assistant and 35 for a customer-facing pricing agent.
Which readiness gap should you close first?
With finite bandwidth, sequence gaps by what unlocks the most.
Where is the highest-leverage readiness gap?
- Data is not accessible to analysts todayClose data accessibility first; without it every other gap is unmeasurable
- No documented use case with a measurable baselineClose use case definition; without B1 to B3, scoring is meaningless
- No executive sponsorClose executive alignment; an unsponsored pilot will be defunded mid-flight
- No PIA or security review completedClose compliance posture; the board will not waive this gate
- No internal capability to operate the systemClose org readiness; see the staffing-gap analysis for sizing
Accessibility lets you measure. Use case definition lets you score. Sponsorship lets you fund. Compliance lets you deploy. Operating capability lets you sustain.
What should the CIO sign off before an AI pilot starts?
When the scorecard supports proceeding, the CIO or CDO answers these 12 questions yes in writing. Anything short of yes is a named accepted risk or a reason to delay.
- Is the scope statement signed by the sponsor with documented stop rules?
- Is the data inventory complete with lineage for every critical element?
- Is the data quality scorecard above the threshold the use case needs?
- Are residency, retention and access controls aligned to Australian obligations?
- Has the Privacy Impact Assessment been accepted by the privacy officer?
- Is the security review complete with no unresolved high or critical findings?
- Is the vendor contract signed with data, liability, exit and SLA terms reviewed?
- Are build, run and oversight roles staffed through the pilot?
- Is the change plan published, with training scheduled and feedback open?
- Is observability live with dashboards, alerts and evaluation hooks?
- Is the baseline measured with the KPI calculation agreed?
- Has the board or executive committee accepted residual risk in minutes?
For the financial case, see our automation payback period guide and AI business case template for boards. For ethics questions, see our AI ethics framework.
What have we learned from building our own AI products?
Building Carbonly.ai and RootCauseAI, our own products, taught us that the technical build is rarely the binding constraint. Data and organisational foundations are. An organisation that does the readiness work moves quickly and recovers from mid-pilot surprises. One that skips it learns the lessons in production.
The assessment produces value on its own: a cleaner inventory, clearer ownership, a sharper use case and a sponsor who understands what they are sponsoring. Below 75, spend two to six months on foundations first. Our data sovereignty guide is companion reading.
How Solve8 can help
Our consultants run pre-pilot readiness assessments as a fixed-scope engagement for a specific use case, or review a scorecard your team has already produced. See our AI strategy service, or contact us to scope an assessment.
