The UK government recently released a report on their cross-department adoption and usage of M365 Copilot with some interesting results. It was an experiment that ran for 3 months across 20,000 government employees (Sept 2024 - Dec 2024) with a main goal being to understand how AI could impact productivity.
Top-Level Insights
From a positive persective, the key findings are as follows:
Average Time Saved: Users reported saving an average of 26 minutes per day using M365 Copilot—equivalent to 13 working days per year.
High Adoption: 80% of the 20,000 participants actively used Copilot, with Teams and Outlook being the most used applications.
Positive Sentiment: 82% of users did not want to return to pre-Copilot workflows. Satisfaction and recommendation scores were 7.7 and 8.2 out of 10, respectively.
The report highlights some areas that were a bit of a challenge, none really surprising and with some focused work could be resolved:
Complex Tasks: Copilot struggled with nuanced, context-heavy, or sensitive tasks (e.g., policy analysis, grievance handling).
Training Needs: Users needed time and support to learn effective prompting and usage patterns.
Inconsistent Experience: Variability in rollout and support across departments affected adoption and outcomes.
Tool Evolution: Mid-experiment updates (e.g., Copilot Agents) were not fully evaluated but generated strong interest.
My Thoughts
So generally, a lot of improvement and positivity from its use which is pretty cool to see with participants saving significant time, roughly freeing up at least 2 weeks worth of time which could be used to focus on other tasks. From the results, it seems content creation, especially first drafts helped a lot. Interestingly, there was some retience among some groups on the appropriateness of using Copilot. I think this speaks to broader AI adoption challenges and hesitation to lean into new things right away without clarity on the guardrails. I thought it was interesting to note that senior civil servants saw the least benefits, which suggests perhaps that AI is the tide that raises all ships and also perhaps those more senior had less time to get engaged with newer tech or even perhaps less inclined to adapt to using AI in work they have been doing for a long time already - change is hard.
In terms of challenges, most could be resolved with better training, better change management and longer adoption time. With certain tasks, this could imply some additional modifications are needed perhaps and could benefit from more curated datasets, task specific tooling and/or task specific agents.
It was interesting to see Teams and Word had the highest adoption of M365 Copilot. Teams makes a lot of sense since users probably spend a large portion of their time there.
Overall, the findings seem to suggest that AI had strong impact here, especially as adoption grew. I like that the UK Government is leaning strongly into AI and their AI playbook is well worth a read.
This recent article in New Scientist highlights how the UK tech secretary (Peter Kyle) uses ChatGPT through a Freedom of Information request to see his prompts. I, for one, am glad to see someone who is responsible for leading the charge on technology is actively engaging with it, even if some of the prompts are not that ground-breaking. We are still at the frontier of this and we need to push the boundaries in many capacities.