This week at work, my team reported a major system outage. There was a lengthy conference call to solve the outage and afterwards someone (not from my team) ran a transcript of the call through copilot to generate an executive summary and sent it up to top level leadership.

Copilot misunderstood the relevance of various comments (surprise!) and effectively reported that my team reported the outage almost an hour after we became aware of it. This caused us to get yelled at by senior leadership for failing to report a major issue in a timely manner. We then had to waste time proving that we had escalated the issue appropriately within minutes and listen to the call recording again to figure out where the incorrect reference came from and provide quotes from the transcript to explain why the summary was wrong.

So far there have been some very small use cases where I have found LLMs slightly useful but mostly they cause me more work than they save (by a lot). And yet, I keep hearing more and more from my company that they want “AI” to analyze things for us.