A Checklist to Troubleshoot KB Effectiveness Issues



Did you make a big investment in your knowledge base, only to find that it’s not nearly as effective as you hoped it would be? This checklist will help you diagnose the problem–and determine how you can turn things around..



  1. How are you defining success?

“What gets measured gets done”, but what if you are measuring the wrong thing? For instance, if you are measuring success by the proportion of users who abort creating a case because an appropriate knowledge base document was presented to them, you are missing out on all the other ways that the knowledge base is helpful—starting with users searching first, before even thinking of creating a case.

Carefully consider how you measure success for the knowledge base. For instance, you can compare knowledge-consuming portal sessions and cases actually logged. (Note that this only focuses on customer usage; the knowledge base is also very useful for the support engineers as well.)

  1. Is your measurement system failing?

Let’s start with an easy check. It could be that everything’s working just fine, but the metrics are incorrect. Check that:

  • The counts are correct
  • The correct formula is applied
  • The reports pick up the right data and format it correctly
  1. Is the deflection workflow actually engaging?

Here’s another easy check. Perhaps your deflection workflow is just not deployed.

  • Can users log cases in non-web channels, bypassing self-service options? (Phone and in-app logging are common offenders.)
  • Is the deflection workflow not working properly? A quick test should confirm that it is.
  • Are the documents not displaying properly? Test and test again.
  1. Are the users to blame?

It may look like users “don’t know” how to use the system—but it’s complicated.

  • Do they use poor search terms? Don’t blame them, update the knowledge base document to use the terminology that they use!
  • Do they fail to open the documents they are shown? Again, address the contents of the knowledge base. If the titles and summaries don’t resonate with the users, they won’t go further!
  • Do they read the “right” documents, but still log cases? You guessed it: the documents are not written in a way that they can be understood and trusted.

Bottom line: don’t blame the users; fix the knowledge base.

  1. Is your search engine insufficient?

We are getting to the good stuff now: are searches returning what they should? You can’t win if

  • The searches retrieve documents that are not relevant.
  • The top matches are not relevant. Most users won’t look past the first few.

If this is the case for you, you’ll need to upgrade your search engine.

  1. Do you need to shore up the knowledge base itself?

We already saw (under #4) that the knowledge base could be the weak point even when it seems that the users are at fault. We are talking here about knowledge holes: users searching for items that simply do not exist.

The good news is that you can gather up all the search terms that lead nowhere, and simply build answers for them.

  1. Are the incentives all wrong?

In some cases, you may be unwittingly encouraging users to log cases rather than self-help. For instance:

  • It’s easier/faster to open a case instead of using self-service options. Is your website up to par? Is it actually faster to get an answer from a human being?
  • Are the support engineers silent about the knowledge base? Once customers realize that they can self-serve instantly from the same knowledge base that the support engineers are using, they will use it more.


Did you figure out what your limitations are? Tell us in the comments (or tell us why you are struggling.)

And if you need help investigating why your knowledge program is not working as you think it should, talk to me.

Leave a Reply

20 − nineteen =

Your email address will not be published.