How to Compute the ROI of AI Projects

You probably have an AI project in the works–or ten, and if so, you are likely wondering about the payoff. Is there something different about AI projects when it comes to computing ROI? What is worth considering in a ROI analysis? What are common mistakes in computing the ROI of AI projects? We explore these questions here.

Let experiments be experiments

I regularly get asked to compute ROI for small-scale experiments. To be blunt, it’s rarely worth it. Instead, embrace conducting experiments on an ongoing basis, for AI and non-AI projects, as a cost of doing business. And, use the experiments to help you decide on what’s worth a larger-scale implementation, and how best to measure a formal ROI for these formal projects. Don’t bother instrumenting the small stuff.

Use your cost model

Beware ad-hoc ROI models that are divorced from your overall budget or cost model. They are likely to lead you astray. Exhibit A would be vendors’ ROI models, which are by nature generic and may not apply to your particular situation–and, to be sure, use the most favorable scenarios for the vendors. So if a vendor suggest that you would save $10 per case by using some new AI feature they are flogging, your first thought should be to question whether the savings would (1) apply to every case and (2) be that high.

If you do not have a cost model, create one. It’s useful for all the business decisions you make, not just the ROI of specific initiatives.

Count all the costs

ROI calculations are very simple: (benefits — costs)/ costs. Per the formula, correctly accounting for the costs is essential as a too small denominator vastly inflates the result. Don’t just focus on the cost of the software, as it’s typically a small proportion of the overall costs. Also include:

• People costs, both for outside consultants and internal staff. Include behind-the-scene players such as project managers, not just the folks with AI in their title
• Training costs, including creating the curriculum and delivering it
• Infrastructure costs such as storage and servers
• Maintenance costs

Don’t worry about tiny benefits

In an effort to show maximum benefits, organizations often create a laundry list of potential benefits and then expand a lot of effort to quantify each and every one of them. It’s a waste of time, and smallish benefits are very hard to forecast and measure accurately in any case. Focus on quantifying the largest benefits, typically one or two, and ignore the others. You can just present them as additional, qualitative benefits.

For Support projects, focus on efficiency gains

Most AI projects in Support result in added efficiency, either by handling customer questions automatically or by helping the support agents resolve issues faster. If your AI project will specifically enable new support options, which you will monetize, by all means compute the expected gain, but otherwise trying to quantify additional revenue generated through a better overall customer experience is elusive, so focus on efficiency gains.

To forecast efficiency gains you need to estimate both the size of the improvement and the scope of it. In other words, are you saving 5% or 10% of the cost of a case, and does that apply to 10%, 50%, or 100% of cases. Unless you are eliminating human-assisted cases entirely, it’s very useful to break down case resolution into phases: initiation, assignment, troubleshooting, engineering handoff, wrapup (your mileage will vary), and assign a weight (cost) to each phase before you forecast the likely savings.

Also, efficiency gains often play out differently for junior and senior agents, between product teams, or between regions. Take such differences into account as you forecast the effect of the changes and as you, later, measure the outcome.

For Customer Success projects, (also) track additional revenue

AI projects in Customer Success may focus on efficiency improvements, in which case the techniques above apply: organize activities by type, break down activities in phases, and apply savings to each phase.

AI projects for Customer Success often have the potential to increase revenue (or decrease churn), and such gains can be very significant indeed. For additional purchases, you will want to track add-on purchases completed or initiated in Customer Success, and perhaps which ones you can attribute to your AI project. Make sure that infrastructure is in place to track the effects you want to measure.

Forecast, then check your work

I often see great effort lavished on creating ROI analyses, usually for the benefit of a CFO who will approve a particular purchase, but very little attention paid six months or a year later, to check what actually happened. Do check! Sure, Support and Success organizations typically have dozens of projects going on at once and it’s hard to identify whether your pet AI project caused a particular improvement, but checking outcomes helps refine assumptions and helps create better models for the future.

How do you go about forecasting and checking ROI? Tell us in a comment.

(And we can help create custom ROI tools. Just ask.)