AI Chatbots: A Use Case & Lessons

This post was co-written with John Ragsdale and summarizes discussions from the Service Leaders Roundtable with Phil.

In our last post, we promised we would share use cases for using AI in support. This is our first one: chatbots.

What’s a chatbot?

A chatbot provides an interactive question-and-answer experience, typically for website users. While early chatbots were rigidly built around preset questions and answers (which, annoyingly, did not match the questions users were actually interested in!), AI chatbots strive to understand the intent of a question, regardless of how it is phrased or spelled, and they “learn” over time what the correct answer is to a question.

Old-style chatbots typically came with an avatar. Today’s chatbots can just be a search box, so users may perceive them more as a search engine.

A Real-World Case Study

This vendor has 16 product families and 100 million users across 400,000 organizations, with about 200,000 customer service cases per year. With lots of non-technical end-users, the volume of simple queries is very high.

After analyzing frequently-asked questions, the vendor chose to focus on a handful of simple questions, mostly customer service and some technical how-tos and chose Bold360 as the tool. The implementation took just 90 days and focused on creating suitable answers for the topics that were both popular and amenable to automation.

The result? A 30% engagement rate and a 31% success rate, much higher than the typical 6% success rate[1]. The vendor dedicated 4 chat agents to handle the situations when the bot cannot provide an answer—a considerable saving from the “before” state.

Lessons Learned

  • Invest in a great support website. An intuitive design and unified search are more important than a chatbot.
  • Make sure you have a good user fit. Chatbots are great for consumers and others who have lots of repetitive questions. Expand (gingerly!) to more complex questions once you have achieved success with the simple stuff. Look for tools that allow you to monitor usage and success.
  • Maintain great content. Chatbots need content to work properly so rev up your knowledge management program. A great bot cannot compensate for a terrible knowledge base.
  • Go commercial. Building your own chatbot is likely to be riskier, more expensive, and take longer than implementing a commercial solution. (Plus, do you want to hire ML experts, conversation engineers, data engineers, computational linguists, and UI designers?)
  • Aim for out-of-the-box. Find a vendor that meets most of your requirements.
  • Offer a seamless transition to assisted support. If the chatbot cannot cope, allow the user to escalate immediately to a live agent (this means you need to have chat support!) And record the interaction, successful or not, so that agents have a full record of the user’s interactions with the bot.
  • Sweat the details. If the website design highlights the chatbot, usage will soar.

Expect a moderate success rate. At this point, chatbots will realistically handle a minority of issues.

Do you have an AI chatbot? Please share your experience in the comments.


[1] The Inner Circle Guide to AI, Chatbots and Machine Learning. ContactBabel 2021

Leave a Reply

Your email address will not be published.


two × three =