AI for internal knowledge
Staff can't find the current procedure across our sites. Can an AI assistant answer from our own documents?
Last updated
Yes. An AI assistant can answer staff from your own procedures, policies and safe work method statements, quoting the source document and version, provided it searches only current approved documents and respects each person's access. Prove it with a test set of real staff questions scored before rollout. Use Microsoft 365 Copilot where documents are well governed in SharePoint, and a custom private assistant where answers must be tightly controlled.
Who this is for
This page is for operations, compliance, WHS and IT leads in multi-site organisations where procedures, policies and safe work method statements sit across SharePoint sites, Teams channels, file shares and document management systems, and staff at each site ring head office or use an old printout because search does not surface the current version.
It covers how a document assistant finds and quotes the right document, how to stop it quoting superseded or restricted content, how to measure its accuracy before staff rely on it, and when Microsoft 365 Copilot is enough. For cleaning up SharePoint permissions before any rollout, see the Copilot oversharing answer linked below.
How a document assistant answers a question
Connect the approved sources
The assistant indexes the libraries that hold controlled documents: SharePoint, OneDrive, a document management system or a quality system export. Drafts, personal folders and archive libraries stay out.
Read the metadata as well as the text
Each document carries its title, version, status, owner, review date and the sites it applies to. Retrieval uses these fields, so a question from a Mackay site returns the Mackay procedure first.
Check access at question time
Before any passage reaches the answer, the assistant checks the person's existing permissions in SharePoint or the source system. Staff never get an answer built from a document they could not open.
Answer with citations
The answer quotes the relevant passage and links to the source document, its version number and section, so staff can open the controlled copy.
Say when it does not know
If no approved document answers the question, the assistant says so and names the document owner to ask, instead of filling the gap from general knowledge.
Superseded documents, site procedures and SWMS
Superseded versions
Only documents marked current and approved are searchable. When a procedure is revised, the old version leaves the index the same day, and every answer shows the version number it used.
Site-specific procedures
Where the same task has a different procedure at each site, the assistant asks which site or uses the person's home site, and states which site the answer applies to.
Safe work method statements
The assistant can find the SWMS for a task and quote its controls. The SWMS stays the controlled document that workers review and sign on site, and the answer links straight to it.
Measure accuracy before rollout
Collect real questions
Gather 100 to 200 questions staff actually ask, from site leads, help desk tickets and toolbox talks, across several sites and roles.
Write the expected answer
Document owners record the correct answer and source for each question, including questions that should return 'no approved document'.
Score every answer
Score each response for correct source, version, site and content. Include restricted questions to confirm the assistant refuses them.
Set the bar before the pilot
The business sets the pass mark, then a pilot group at one site uses the assistant on live questions while results are compared with the test set.
Re-run after every change
Re-run the test set when documents, models or settings change, and keep the results for audit.
Microsoft 365 Copilot or a custom private assistant
| Question | Microsoft 365 Copilot | Custom private assistant |
|---|---|---|
| Where the documents live | Mostly SharePoint, OneDrive and Teams | SharePoint plus other document or quality systems |
| Which documents it answers from | Anything the user can open, unless restricted | Only the libraries and statuses you approve |
| Version and status control | Depends on how libraries are governed | Filters on version, status and site metadata |
| Answer format | General assistant with citations | Fixed format showing source, version and site |
| Accuracy testing | Manual test sets | Test set re-run on every change |
| Best fit | Well-governed Microsoft 365 with broad office use | Frontline or regulated procedures where answers must be controlled |
Keeping the data in Australia
A custom assistant can run in your own cloud environment in an Australian region, with the index, logs and model calls kept onshore and no company data used for model training. For Microsoft 365 Copilot, Microsoft states that prompts, responses and Microsoft Graph data are not used to train foundation models, that Copilot is covered by the data residency commitments in its Product Terms, and that customers outside the EU may have queries processed in the US, EU or other regions. Confirm the position for your Microsoft 365 environment before documents holding personal information are included.
The OAIC's guidance on commercially available AI products asks organisations to do due diligence, test accuracy, keep human oversight and consider APP 8 where personal information could be disclosed offshore. In New Zealand, the Privacy Act 2020 applies.
Limits
Weak document control
If nobody knows which version is current, the assistant cannot know either. Fix status and ownership on the most-used libraries first.
Scanned documents
Old scanned procedures need text recognition and checking before they are searchable, and some are better rewritten.
High-risk work
Answers support understanding. Isolation, confined space and other high-risk work still follow the site's permit and sign-off process.
How Solve8 delivers it
Solve8 works in fixed-fee phases: discovery on one document set at one site with a measured baseline of how long staff take to find answers today, a shadow pilot on live questions where you set the accuracy bar, a build into your Microsoft 365 and document systems on your infrastructure in Australia, then support or handover. Document owners approve changes to sources. Workshops run on site and builds run remotely under your access controls.
Related work
- Copilot oversharingThe permissions clean-up to run before any AI reads SharePoint.
- Private AI infrastructureAI models and indexes hosted in your own environment in Australia.
- Copilot readinessFind what Copilot would surface and design a measured pilot.
- AI pilot to productionHow to move a pilot into daily use with a measured accuracy bar.
Sources
Microsoft Learn, Data, Privacy, and Security for Microsoft Copilot: https://learn.microsoft.com/en-us/copilot/plan/microsoft-365-copilot-privacy
Office of the Australian Information Commissioner, Guidance on privacy and the use of commercially available AI products: https://www.oaic.gov.au/privacy/privacy-guidance-for-organisations-and-government-agencies/guidance-on-privacy-and-the-use-of-commercially-available-ai-products
Questions
Will a procedure assistant only quote documents each person is allowed to open?
Yes, if it checks each person's existing permissions when they ask. Microsoft states Copilot only surfaces content a user can at least view, and a custom assistant should apply the same check. Old sharing links and broad grants still expose content, so fix the riskiest sites first. The permissions clean-up itself is covered in the Copilot oversharing answer.
How do we stop an AI assistant quoting an old version of a procedure?
Index only documents with a current, approved status, remove superseded versions from the index when a revision is published, and show the version number in every answer. Put questions about recently revised procedures in the test set so stale answers show up before staff see them.
How accurate does a document assistant need to be before rollout?
The business sets the bar per question type before the pilot. Correct source and version on routine questions, and a refusal when no approved document applies, are the usual tests. Set a higher bar for safety and HR questions and name an owner who reviews each failure.
Should we use Copilot or build a private assistant for our procedures?
Use Microsoft 365 Copilot if procedures live in well-governed SharePoint and staff already hold licences. Build a custom private assistant when answers must come only from controlled documents, frontline staff without Copilot licences need access, or key documents sit in systems outside Microsoft 365.
Who can build an AI assistant over company documents in Sydney, Melbourne or Brisbane?
Microsoft partners can configure Copilot and SharePoint. For a private assistant across SharePoint and other document systems, with version control, permissions and a scored test set, use a firm that builds and tests retrieval systems. Solve8 runs workshops on site in Australian cities and builds remotely under your access controls.
Related
- We bought Microsoft 365 Copilot licences. How do we make sure it doesn't show staff files they shouldn't see?Microsoft 365 Copilot only shows what staff can already open, so oversharing is a permissions fix. Reports, labels and Purview steps to run before rollout.
- Copilot readinessPermissions and data ready before rollout.
- Private AI infrastructureAI that runs inside your own environment.
- Data sovereignty in Australia: a practical guideWhat Australian law requires about where data lives, the sector rules that go further, and how to choose AI vendors with confidence.
- Custom AI developmentAI that reads, checks and drafts the first pass.
- Our AI pilot worked in testing but nobody uses it six months later. What went wrong, and should we fund the next phase?Why an AI pilot that worked in testing goes unused months later, how to find the cause, and how to decide whether to fund the next phase, fix it or stop.
Procedures spread across sites and systems?
A 30-minute review with a senior consultant. You leave with a written one-page summary you can share internally.
Fixed-fee phases, agreed before we start. Stop after any phase.