Microsoft Copilot 365 - The good, the bad, and the hefty price tag

All aboard the AI hype train

I'm a technologist, and as one of those, of course I'm hugely excited about AI. Naturally, when the latest bit of AI tech drops I have to get my hands on it. I'm eager to see if I can incorporate it into my workflow, and gain any clues about how the rise of machine intelligence is going to affect us all as it improves.

I'm also a business owner, with a business built around our managed IT service, of which Microsoft 365 is a critical component. So, as you might guess, when Microsoft 365 copilot was announced, I was hyped.

Rolling back the years. Thanks ChatGPT!

What even is Microsoft Copilot?

If you're not sure what Microsoft 365 Copilot is, first of all I don't blame you. In typical Microsoft style, the team in charge of naming seems to have tracked down a group of the most mercurial minds on Earth to make the name as complicated as possible.

The name Copilot was first used by Microsoft when they launched GitHub Copilot, a software development assistant, but as the rush to pump out in-demand AI tools has progressed, we've ended up with Copilot in Bing, Copilot in Windows, Copilot for Sales, Copilot for Service, Copilot in Viva, a newly announced Copilot PC, and of course Microsoft 365 Copilot.

All the other flavours of Copilot are designed around specialised workflows, but the exciting one for most people is Microsoft 365 Copilot. This version integrates directly into Windows, and extends the familiar chat interface by giving it access to all your emails and documents, as well as integrating directly into office apps like Outlook, Teams, and Excel. 

In real word terms, that means rich functionality, like the ability to pull up a chat window and say "Yo Copilot, how much was the invoice I got for pizzas last week?" and have it search through your emails, look in the attachment, and tell you how much you owe the pizza man. In apps, you've got buttons to generate charts from spreadsheets with a single click, draft emails, or reword sections of word document.

Copliot, take care of this for me please, I've got more pizza to order.

Sounds cool but is Microsoft 365 Copilot worth the cost?

All exciting stuff, and features that have the ability to make the working day that little bit easier for those of us that use PCs in a professional setting. We can't go any further however without mentioning the elephant in the room (or elephant in the title of the post?) ... the cost.

Microsoft 365 Copilot in the UK is currently just shy of £300 for a single licence. You have to pay for an entire year up front, and you'll have to fork out for every user that wants the functionality. Being from Yorkshire, I have a geographic disposition against spending money. Or in other words, I'm a tight git. If something costs that much, it better be worth it, and I'm sure there are many businesses up and down the country weighing up the cost.

Determining how we help our customers weigh up the cost is something we've found tricky. I've got a licence, they can have a demo from me, I can show them round, show them how it all works. They can even purchase a small quantity of test licences and distribute those to key people within their company to help them understand the product.

The problem, however, is that getting the most value from Copilot is an entirely personal endeavour. It's a bit Marmite. Some people will love the content generation features, others will drown in a sea of frustration over inaccuracy and detail. 

We've found then, the best thing to do is talk about our own experience with the product, across a variety of tasks, and explain where it shines, and where it falls short, and hopefully provide realistic hype-free information that can help make informed decisions.

So after using Microsoft 365 Copilot for around 6 weeks now to help with my own workload, here's what I think.

The good things about Microsoft 365 Copilot

Better search

Finally! I'm sure I'm not alone in saying my memory for work related things is truly awful. I'm often asked for specifics on things that happened months ago and draw a blank. To combat this, I have a healthy habit of writing things down, but digging that info back up can often be a nightmare. 

Not with Copilot though. Having the power of a large language model all up in your stuff really does feel like you have superpowers. Copilot doesn't just have access to the contents of all your documents and emails, it understands them too. Instead of trying to remember exactly how you phrased that one clause in a customer proposal, you can skip to the root of the problem and ask things like "does customer X have different terms to customer Y" with surprisingly good results.

Integration into Office apps and Windows

You'd think with a hefty price tag, there had better be something to show for it, and in my opinion this is where much of the effort has gone. Activating Copilot 365 is like turning on a whole new mode for Windows. After switching it on, you suddenly get extra buttons in apps, prompts to summarise emails, and whole new sidebars in apps packed with functionality.

It makes sense too, the selling point of Copilot is that it goes beyond the chatbot style interface, and gives users helpful tools that work at the click of a button, requiring no explaining or prompt engineering. 

Whether you like the output of all these features, and whether these features are ready for business use is still up for debate, but you can't fault the effort and ambition Microsoft has put into the product. The coverage across apps, depth, and attention to users' needs is nothing short of impressive.

Data peace of mind

This is worth mentioning because some of the practices used to make AI better are a little sketchy. The fear is if AI consumes proprietary content, it could potentially regurgitate company secrets to other users. If you're a business working in a particularly niche field, working on pushing industry boundaries, you'd be more than disgruntled if all your hard work was available to anyone with access to ChatGPT.

With many huge corporate customers, it's no surprise then that Microsoft takes this seriously, and their promise is essentially that your data won't leave your organisation. For many, this doesn't just make Copilot the preferred AI tool, it makes it the only option.

Copilot lowers the barrier to learning

There are some features in Copilot that allow users to perform beyond their skill set. Whether it's generating advanced formulas in Excel, producing rich graphical presentations in PowerPoint, or altering the tone of an email to better suit a different audience. Now behind a simple click of a button, these things are possible for users who might not otherwise have the knowledge or experience to do them manually.

It's important to stress that the output does often need checking over, but the very fact that users can generate high quality contextual examples of work, I think is a great learning aid. 

Easier interactions

Another thing I'm openly very bad at is scheduling meetings in almost any calendar type application. For some reason, having four or five form fields I need to set almost guarantees I'll make a mistake. It's got to the point where an apology to the client for scheduling a meeting on the wrong day is almost routine for me.

It's nice then to have an alternative way to achieve those click heavy tasks in the form of the "digital assistant" part of Copilot. I can click a button on my taskbar and even talk to the thing to get a meeting arranged. I don't think it's necessarily a quicker way to do things, but it's for sure a more natural, less error prone route for me.

The bad things about Microsoft Copilot 365

Integration into Office apps and Windows

Yeah, I'm listing this one as both a good thing and a bad thing. See, the advantage of the chat based interface is that you can have a conversation about the work you're doing, offer feedback, make tweaks, and iteratively improve whatever you're working on.

When you're in Word for example, you can either generate content from a much smaller length limited prompt, or right-click on a block of selected text to redraft it in similar but different words. Both are one shot things, and you don't have the opportunity to feedback on the generated content. There is a sidebar chat interface in Word, but for me, this seemed to be more geared around asking questions about the doc and summarising information than it was generating content.

It's not as good as ChatGPT

Luckily, Copilot does come with a chat feature, so if you do want to iteratively shape a piece of generated content there is the option of interacting with it in the traditional manner. What surprised me however was how far behind the competitors Copilot chat seemed to be. I'd often find myself reminding it about instructions at the start of a conversation, something that just doesn't seem to happen when I'm using ChatGPT or Google Gemini.

Copilot chat also had a tendency to lurch towards my documents and emails at every opportunity, so if I was doing a content checking task, we'd often be derailed by it pulling up a document from years ago, and have to battle to stay on track.

This is something that should improve over time, and given how much better ChatGPT currently is, and how closely partnered they are with Microsoft, you'd expect the advancements to land in Copilot sooner rather than later. At the time of writing however, it does need to be mentioned the asking price is for a model that is a long way behind the competition.

Option paralysis

As previously mentioned, the Copilot integration into Windows and the Office apps is thorough, and they've really tried to tie certain tasks into simple routines that are easily accessible. For broader tasks outside this pre-defined training route however, it can be hard to know where to go. Let's say you want to draft a post for social media. You can open Word, Loop, OneNote, the assistant in Windows, or the one at chat.office.com, or even open Edge and pull out the Bing sidebar there. All of them will work, but all give slightly different results, so which is best?

Again, this may improve over time. The unified place you can go to start any task seems to be the holy grail for feature development in 365, so if Microsoft can pull it all together and make Copilot that place, they'll be onto a very good thing. Until then, it's still on us to figure out what works best.

Mistakes and the devil in the detail

Underneath all the hype, the marketing, and the product integrations, Copilot is mostly powered by the same large language model technology that tools like ChatGPT are. This means it's still prone to mistakes and can even sometimes make things up entirely. The example above where the user is asking Copilot to generate a proposal based on meeting notes frankly terrifies me.

Running that particular gauntlet, there's potential for big mistakes in pricing, or over promises on service delivery. Fairly easy to spot and correct, it's the minor details, like a tendency to fall back on US spelling and grammar conventions even when told not to that are much harder to spot, and much more time consuming to edit. 

For me, the editing became almost as time consuming as generating content manually, narrowing the use case, and squeezing the value out of that £300 I paid for my license. 

Final thoughts

There's more than just potential in Microsoft Copilot 365; there are a number of features that are already well developed, and will ultimately help people to become more productive at work. It's definitely not snake oil, and there's definitely good reason for all the surrounding hype. It is however, very hard for me to recommend it.

If you're interested in using it, I would suggest testing out the free version first. Press the Copilot button on your Windows taskbar, or if that's not available, head over to Bing.com and click on the Copilot button there. Give it a task to do, and get it to simulate writing an email for you, or drafting up some content for a proposal. If you're satisfied with the output, if you're finding it useful, progress to the next step either schedule a demo with someone who has a licence (hey that's me 👋), or buy a full licence for yourself, and test out the integrated features. Only then, if you're still happy, would I recommend looking into rolling it out across a whole organisation. 

My advice is to evaluate properly. Take it one step at a time, and resist any urge to splash out just because it's the latest new thing (spoken like a true Yorkshireman).

If you need any help at all, no matter how far along you are in your Copilot discovery process, please don't hesitate to get in touch. All of us here at Impelling are happy to help where we can, and will provide realistic practical advice without any obligation to buy our services.