Microsoft Copilot ROI: Which Teams It Helped
Know Which Teams Copilot Changed Work For
Six months after a Microsoft 365 Copilot rollout, the useful measure is how long named tasks take now, and how much of the output gets reworked, team by team. The active-user count in the admin centre will not tell you that. It tells you who tried the tool.
Run the review on a handful of tasks that sit behind a queue: the monthly board pack, tender responses, meeting minutes, policy drafts, customer correspondence. Time them against a baseline, count the edits, and sort licences by role. You end with a defensible renewal decision and a short list of the tasks where a purpose-built agent would do more than a chat window.
This guide is for the IT manager, finance lead or operations manager who signed off the licences and now has to explain what they bought. The usual trigger is a renewal date, a CFO asking for the number, or a first AI experience that fell flat.
Working out whether this fits your business? Book a free 30-minute review
What the Admin Centre Counts, and What It Leaves Out
Microsoft's own documentation is precise about what its headline figure means. In the Microsoft 365 Copilot usage report, "active users" are enabled users who tried a user-initiated Copilot feature in one or more Microsoft 365 apps over the selected timeframe. One summary request in Outlook in the last 28 days counts the same as a finance analyst drafting every variance commentary with it.
That is a reasonable thing for a licensing report to count. It is a poor proxy for value. An active-user rate of 80% can sit alongside no change at all in how long the board pack takes, because the board pack was never where people used it.
The Copilot Dashboard in Viva Insights adds more detail, such as usage by app and by group. It still measures the tool. The question a renewal needs answered is about the work: did a specific task get faster, better, or neither, and for whom.
What the Public Trials Found
Three government evaluations are public, and each is more careful than most vendor case studies. They are worth reading before you write your own review, because they show where self-reported savings and measured productivity part ways.
The Digital Transformation Agency ran a whole-of-government Microsoft 365 Copilot trial in the first half of 2024. Its summary of evaluation findings reported perceived time savings of around an hour a day on summarisation, first drafts and information searches, with about one in three participants using Copilot daily. The same evaluation noted that Copilot's unpredictability and lack of context meant time spent verifying and editing output, which negated some of the efficiency savings.
The Treasury ran its own 14-week trial, evaluated by the Australian Centre for Evaluation and published in February 2025. Staff found Copilot helpful for basic tasks but less useful than they had expected. The evaluators estimated that a staff member at APS6 salary would need to redirect about 13 minutes a week to higher value work to offset the licence cost, and noted that this saving was not quantified in the trial.
The UK Department for Business and Trade gave out 1,000 licences between October and December 2024. Its evaluation report, dated August 2025, found 72% of users satisfied and the largest time savings on written work. Some other tasks took longer with Copilot, and the evaluation did not find evidence that the time savings had led to improved productivity.
What the published Copilot evaluations reported
Read together, the trials say something consistent. People like the tool, and they believe it saves time on writing. Where an evaluation went looking for that saving in the output of the work, it was hard to find. A six-month review inside your business should be designed so that it can find it, or show plainly that it is not there.
Choose the Tasks Before You Measure Anything
The review starts with a short list of tasks, not a survey. Pick three or four that meet three tests: they recur at least weekly, someone waits on the output, and you can say what a finished one looks like.
Good candidates in most Australian businesses:
- The monthly management or board pack, from data pull to commentary
- Tender and proposal responses, especially the repeated compliance sections
- Minutes and action lists from recurring meetings
- First drafts of policies, procedures and customer letters
- Searching past project files, contracts or correspondence for a specific answer
Leave out anything that happens once a quarter, because six months gives you two data points. Leave out anything nobody owns, because there will be no one to agree the baseline with. If the task list is hard to agree, the AI process audit method of scoring by volume, judgement, data access and the cost of a wrong answer is a quick way to settle it.
How the Six-Month Review Runs
The review compares the same task done with and without Copilot, by people who do it for a living. It takes a few weeks, most of it waiting for tasks to come round.
How a six-month Copilot review reaches a renewal decision
Time the task, end to end
Measure elapsed working time from the moment the task starts to the moment the reviewer accepts it. Include the checking and editing. The DTA evaluation found verification ate into the savings, and a review that times only the drafting step will overstate the gain every time.
Time a small sample of each task both ways. Five to ten instances per task is usually enough to see whether there is a difference worth discussing. The aim is a decision, not a research paper.
Count the rework
For each output, the person who approves it records how much they changed and why. Three categories cover most of it: factual errors, missing context the drafter should have known, and tone or format. A task where Copilot saves 20 minutes of drafting and adds 25 minutes of fact-checking is a net loss, and only the rework log will show it.
Ask about the queue
Ask the people waiting on the output whether it arrives sooner. A finance team that drafts commentary faster but still sends the pack on day eight has changed nothing the board can see. If the queue did not move, find the step that holds it. It is often a data pull or an approval, and no writing assistant touches either.
What the Numbers Usually Separate
Once the tasks are timed, the results tend to fall into three groups. The names and figures in the table below are illustrative, to show the shape of a finding. Your own numbers come from your own baseline.
Reading a six-month review: illustrative findings by task type
| Metric | What the usage report shows | What the task review shows | Improvement |
|---|---|---|---|
| Meeting minutes and action lists | High usage in Teams | Drafting faster, light rework | Keep |
| Tender compliance sections | Moderate usage in Word | Faster first draft, heavy fact-checking | Needs grounding |
| Monthly board pack | Low usage in Excel | No change, data pull is the delay | Different fix |
| Policy and letter drafts | Moderate usage in Word | Faster, tone edits only | Keep |
| Searching past project files | High usage in Copilot Chat | Answers found, some from stale files | Clean the source |
The first group is where Copilot already pays its way: short writing tasks where the drafter knows enough to catch a wrong sentence quickly. Keep those licences.
The second group is where Copilot saves drafting time and then gives it back in checking. That usually means it is drafting without the right source documents. The fix is either better grounding (a curated SharePoint library, sensitivity labels, a cleaner permission model) or a purpose-built agent that drafts only from approved content and shows its sources. Our Microsoft 365 Copilot readiness audit covers the permissions side, which matters again here, because a Copilot that finds stale or overshared files produces confident answers from the wrong documents.
The third group is where nothing changed because the delay was never in the writing. A board pack that waits on three systems to be exported and reconciled needs integration or a workflow, and a chat window will not fix it. That is a Power Automate or agent decision, and it should be scoped as its own piece of work.
Deciding What Happens at Renewal
With the review done, the renewal stops being a single yes or no. It becomes a decision per role and per task, which is easier to defend to a CFO and easier to revisit in another six months.
What to do with each role's licences at renewal
Two cautions apply. First, low usage is not always low value. A contracts manager who uses Copilot twice a month to search a year of correspondence before a dispute may get more from it than someone who uses it daily for emails. The task review catches that and the usage report does not.
Second, check what people are using instead. If licences are dropped and the work still needs doing, staff tend to paste the same content into a free consumer tool. That is a data handling problem under the Privacy Act 1988 for any business holding personal information. Our guide to shadow AI already in your business covers the inventory and the policy that should sit alongside any licence change.
Where a Purpose-Built Agent Beats the Chat Window
The review often surfaces one or two tasks where Copilot is close to useful but not quite there. The pattern is consistent: a recurring task with a known set of source documents, a fixed output format and a reviewer who checks every one. Tender compliance answers, contract clause comparisons, standard customer responses and month-end commentary are common examples.
For those tasks, a purpose-built agent inside your own Microsoft 365 tenant can draft only from the approved library, cite the clause or document it used, and hand every output to a named person to approve. It runs in shadow mode first, beside the team, with each draft compared against the human version until it meets an accuracy threshold you set. Nothing reaches a customer until it does.
That is a scoped piece of work with a baseline and an exit point, not another licence. Our AI strategy consulting engagement starts there: the task review becomes Phase 1, and the one or two tasks that earn a build carry straight into it.
A Practical Review Timeline
A six-month review fits in about five weeks of elapsed time. Most of it is waiting for weekly and monthly tasks to come round.
Six-month Copilot review
The output is a short renewal paper: the tasks measured, the result for each, the licence decision by role, and any task that warrants a scoped build. Keep the baselines. Next year's renewal will be faster because the comparison already exists.
Who Runs It, and Where Help Fits
Most of this review can be run internally by whoever owns the Microsoft 365 tenant, working with the managers of the teams involved. Where it helps to bring someone in is the task selection, the rework categories and the judgement on which tasks need grounding, integration or an agent. That judgement is the part that changes what you spend next year.
The lead on every Solve8 engagement spent 18 years on operations and data systems for BHP, Rio Tinto and Senex Energy, gained while working with previous consulting employers. These companies are not Solve8 clients. The habit that carries over is to time the work before deciding what to build. We are based in Brisbane and work across Australia; if you are local, our AI consulting in Brisbane page covers how we work on site. For a second view on how an independent adviser fits alongside your Microsoft partner, see our guide to AI advisory services in Sydney, which applies in any city.
Common Questions
How do you measure Microsoft 365 Copilot ROI after rollout?
Pick three or four recurring tasks that someone waits on, time them end to end with and without Copilot, and have the reviewer log every correction. Compare time, rework and whether the queue moved, role by role. The active-user count in the admin centre shows who tried the tool, not whether the work changed.
What does active users mean in the Copilot usage report?
In the Microsoft 365 admin centre, active users are enabled users who tried a user-initiated Copilot feature in one or more Microsoft 365 apps during the selected period. A single use counts, so a high active-user rate can sit alongside no change in how long any task takes.
What did the Australian Government Copilot trial find?
The DTA's whole-of-government trial in the first half of 2024 reported perceived time savings of around an hour a day on summaries, drafts and searches, with about one in three participants using Copilot daily. It also found that verifying and editing output negated some of those savings.
Should we cancel Copilot licences that are rarely used?
Check the task first. Some roles use Copilot rarely but for high-value work, such as searching a year of correspondence before a dispute. If no recurring task is found, reassign or drop the licence, and check that staff are not moving the same work into a free consumer tool.
When is a custom agent better than Copilot?
When a recurring task has a known set of source documents, a fixed output format and a reviewer who checks every one. A purpose-built agent in your own tenant can draft only from approved content, cite its sources and run beside the team until it meets the accuracy threshold you set.
Get Your Renewal Answer
If the Copilot renewal is coming up and the only evidence is a usage chart, a task review gives you something a CFO can sign. It ends with a licence decision by role and a short list of tasks worth building for.
Book a 30-minute scoping call and bring the three tasks your teams say Copilot helps with most. We will tell you how we would measure them. Where the review points to permissions and grounding work first, our Copilot readiness service picks it up.