What you should be asking about AI

In partnership with

I sat in on a board meeting last year, where the CTO spent 30 minutes showing off her new AI customer service bot. The demo was flawless, and the board was impressed. The bot answered complex questions, handled edge cases, and even cracked jokes.

Three months later, customer satisfaction scores had dropped 15%.

What happened? The AI could answer questions brilliantly, but it couldn't escalate appropriately. It couldn't read the emotional subtext when someone was truly frustrated. And it couldn't handle the angry customer who just wanted to talk to a human.

The board had asked all the wrong questions: "How accurate is it?" "What are the cost savings?" "When will we see ROI?"

They never asked: "Can you walk me through how a frustrated customer gets help?"

The 5 Questions You Should Be Asking

1. Can you explain how this AI made that decision? Not the technical details, but the business logic. If your AI denies a loan, recommends a price change, or flags a transaction, can someone explain why in terms your customers would understand? If not, you're building a black box that'll bite you when something goes wrong.

2. What happens when this AI is wrong? Every AI system fails. The question isn't if, but when…and what's your fallback? I've seen companies deploy AI without any error-handling process, then scramble when the inevitable happens. Smart leaders plan for failure up front.

3. Who owns the decision when AI is involved? This is where most implementations fall apart. The AI recommends, but who decides? Who's accountable when that recommendation leads to a problem? Without clear ownership, you get finger-pointing and paralysis when things go sideways.

4. How do we know this is solving the right problem? Just because you can automate something doesn't mean you should. One client spent six months building an AI system to optimize their supply chain, only to realize their problem was poor vendor communication. The AI worked perfectly…on the wrong problem.

5. What are our people doing instead? AI should free up humans for higher-value work. But if you haven't defined higher-value work, you've just created expensive automation. The best AI implementations have a plan for what people will do with their newfound time.

The Real Test

These questions reveal whether you're doing AI or just playing with it. AI implementation is messy, requires clear accountability, and always includes a human element. AI theater looks impressive in demos but crumbles under real-world pressure.

Your job as a leader isn't to understand the technology but to ensure the technology serves the business. These five questions will tell you whether your AI investments drive real value or if your AI investment has been a waste.

If you're struggling to have honest, valuable AI discussions within your company, this is exactly what I help leadership teams navigate. Reply to this email to discuss shifting these conversations from fun demos to business impact.

PS - I've developed a simple framework that helps leaders ask the right AI questions before they invest. It takes 5 minutes and has saved many clients from expensive mistakes. Reach out if you want me to share it.

If you found this post helpful, consider sharing it with another executive grappling with AI, technology, and data. If you want to explore AI and other technology strategies, grab some time on my calendar, and let's chat.

The newsletter every professional should be reading

There’s a reason Morning Brew is the gold standard of business news—it’s the easiest and most enjoyable way to stay in the loop on all the headlines impacting your world.

Tech, finance, sales, marketing, and everything in between—we’ve got it all. Just the stuff that matters, served up in a fast, fun read.

Look—over 4 million professionals start their day with Morning Brew’s daily newsletter, and it only takes 5 minutes to read. Sign up for free and see for yourself!

Check it out

Newsletter Recommendations