The FT Word – June 2007

The FT Word

The FT Word is a free monthly newsletter with support management tips. To subscribe, send us email with your email address. The subscription list is absolutely confidential; we never sell, rent, or give information about our subscribers. Here's a sample.

Welcome

Welcome to the June 2007 edition of the FT Word. Forward it to a colleague!

Topics for this Month

Best practices for Sustaining Engineering: a fitting follow-up to last month’s feature about using support data to push improvements in the product
Leading vs. lagging indicators
FT Works in the news: congratulations to Mentor Graphics for winning an ASP “Best Sites of 2007” award

Best Practices for Sustaining Engineering

Thank you to Stephen Laroche for suggesting this topic.

Even when you think your support organization is, to quote Mary Poppins, practically perfect in every way there comes the ugly reality: Sustaining Engineering. Since some percentage of customer issues requires Engineering involvement, your work is not done until the entire process, including the Engineering loop, functions properly. Depending on another team for assistance is always tricky, all the more if you support complex products or if the Engineering organization is dispersed around the world. Here are some thoughts on how to improve the situation for you and for your customers.

1. Define why you need Engineering assistance

Tech Support teams need Engineering in two circumstances: to fix product defects (bugs) and to tackle customer issues that are too complex for the skills and knowledge of the support engineers.

Fixing product defects is usually best done within the Engineering team: it exercises a different set of skills (writing code rather than troubleshooting issues) and it often requires an in-depth understanding of the code as a whole, which precludes amateur efforts. Some support teams choose to create an internal (to support) code-fixing unit, but it’s rarely a successful endeavor, as the code-fixing unit is too separate from the code-creating unit to be able to interact productively. (Even dedicated Sustaining teams within the Engineering group have that problem!) Many support teams, however, include at least a few individuals who can read code, answer customer questions that require that level of knowledge, and even pinpoint causes and fixes for bugs as they transition the issue to Engineering.

Resolving complex issues is much less clear-cut since it’s always difficult to draw a line between a Support-only question and a question that requires Engineering. As mentioned above, many support teams are successfully going beyond pure black-box support (without looking at the code) but even those need Engineering help on occasion.

It’s very helpful to categorize requests to Engineering as either bug fixing or technical assistance (even though we know all too well that a suspected bug is not always a bug.)

2. Minimize your dependence on Engineering

It’s always easier when you control your own destiny. Even if you happen to work with a very accommodating Engineering team, always be on the lookout to minimize your dependence on it by increasing the technical skills of your team. This is true both for bug fixing and for technical assistance.

For bug fixing: instead of dumping vague descriptions of suspected bugs into the Engineering queue, create a crisp reproduction of the problem complete with a script or whatever other method is applicable to your product environment. For complex situations save the entire reproduction environment for use by Engineering. Clearly this increases the burden on Support, both in time and in the skills required of the support engineers, but it also makes it much easier for the Engineering team to quickly get up to speed on problems – without having to go back to the customer weeks or months later for more details!
For technical assistance: a robust knowledge management system is your best line of defense to build up your internal knowledge, and so is the use of a few leads or point people who make sure that each assistance request is indeed necessary. As we know more layers is not a good idea for effective support but some level of filtration is needed here.

Expect to pay more for the talent needed to be independent, and also expect to spend time developing people. New support organizations with complex products may take years to (mostly) wean themselves off Engineering.

3. Create a process and SLA

I have fond memories of working in startups where getting Engineering assistance meant moseying on to a nearby cube, greeting its occupant by name, and usually getting an answer on the spot. (It almost made up for the concentration of bugs typical in immature products!) Once the team gets larger, allowing everyone to interrupt everyone is nuts – plus you really want to track what’s going on.

Track requests through the case-tracking system, the defect-tracking system or a combination. My preference is to track all (suspected) defects through the defect-tracking system, which means allowing at least some support engineers to enter bugs – preferably all of them, or all level 2 engineers. If they turn out not to be bugs, so be it. This is why we invented the “not a bug” status. I like to track assistance requests through the case-tracking system, but it’s usually a bear to get Engineering to use it: know when to concede the battle so you can win the war. Whatever you do, make sure you can (1) check the status of any request in real time and (2) run metrics.
Define SLAs. Be warned that Engineering organizations are extremely wary of SLAs so that a copious amount of reassurance that it’s good to have a target, even if they don’t hit the target 100% of the time. To set the targets think as a customer: if you hit a suspected bug, when would you like to get a confirmation that it is indeed a bug? An hour? A week? A month? Most organizations do well with a two-level SLA: one for emergencies (system down), which can be an hour, around the clock; the other for significant issues, which can be a business day to a week. There is rarely an SLA for minor issues such as cosmetic issues, which may never be addressed outside major revisions. Achieving the SLA should be a significant part of the Engineering team’s goals. Most Sustaining Engineering disasters can be tied directly to the fact that the Engineering team’s one and only goal is to meet release targets.
Think through the mechanics of providing assistance. Especially if the Engineering team is dispersed, simply finding a time in the day to talk can be a significant challenge, which is yet another reason why a tracking tool is essential. At least for critical issues it makes sense to have a few Engineering resources positioned in roughly the same time zones as customers.
Don’t forget personal relationships. I have personally witnessed two complete turnarounds of a war-like situation between Sustaining Engineering and Support due to the personal influence of a wonderful manager I put in charge of that relationship, and both of them happened thanks to the way the manager was able to forge good relationships with the Sustaining team and encouraged the rest of the group to follow suit.

4. Measure and adjust; repeat

Track the Engineering metrics as carefully you would Support metrics, including

volume (new/backlog/closed)
response time (against SLAs)
resolution time (again, against SLAs)
percentage of cases linked to defects. This is one of the key customer satisfaction metrics. For mature products you want to be at 5% or less. Large numbers of bugs are, in my experience, the #1 reason for extreme customer dissatisfaction. Be patient with new products but within a few maintenance release you should see a significant increase in quality.
percentage of technical assistance requests versus bug fixing requests and percentage of assistance requests that were justified. These metrics are indicative on how well the support organization is doing on step #2. There should be very few technical assistance requests (say, fewer than 10% of all requests to Engineering) and virtually all of them should be justified (that is, not something already documented in the knowledge base or otherwise). There may need to be some “negotiation” around what constitutes a justified request – but that’s why you will have the strong personal relationship in place

Meet at least monthly with the Engineering team to review progress and make changes. It’s rare to hit on a perfect process throughout on the first try, and adjustments for new products and new circumstances are always required.

Leading and Lagging Indicators

I got thinking about leading and lagging indicators last month as I was working with a customer that is working on backlog and is struggling to define a solid staffing model.

For planning purposes and to measure success long-term, we rely on a number of lagging (after-the-fact) metrics such as

customer satisfaction ratings
closed cases and productivity (cases closed/head)
percentage of cases linked to bugs
renewal rates

However, to manage the team day-to-day lagging indicators come too late. By definition, customer satisfaction surveys are done after the fact, so if customers are steaming today you won’t know until some time later, long after the damage is done. So you need to also look at leading indicators, metrics that would indicate trends. Here are a few examples:

escalation volume: always a good indication of customer satisfaction – although as we know escalation volume is extremely volatile and a couple of bad customer situations does not indicate a crisis per se
incoming case volume: volume spikes are rightly dreaded by support managers – although we’ve all seen how support teams can rise to the occasion and pick up the pace, to some degree, to match busy periods
backlog: in my mind, a not dreaded-enough, simple metric. Backlog is dangerous in two ways: the support staff gets bogged down when faced with too many open cases, and naturally customer satisfaction suffers.
Engineering handoff volume: a perfect leading indicator of bug volume (but it could also mark the loss of talent within the support team)
new documents in the knowledge base: a neglected knowledge base will die eventually so a steady volume of new documents is healthy, except for completely mature products.

FT Works in the News

One of my web self-service customers, Mentor, won an ASP award for best web site of 2007. (Yes, I was a judge for the award, but I was not assigned to review Mentor…) Congratulations! You can see more details at http://www.asponline.com/07awardsPR.html

I was delighted to see a number of readers at the SSPA conference in San Diego last month, including at the presentation about ROI I gave entitled Money and the Web: Making more and spending less through online support. If you missed it but would like a copy of the presentation you can drop me a note.

Curious about something? Send me your suggestions for topics and your name will appear in future newsletters. I’m thinking of doing a compilation of “tips and tricks about support metrics” in the coming months so if you have favorites, horror stories, or questions about metrics, please don’t be shy.

Regards,
Françoise Tourniaire
FT Works
www.ftworks.com
650 559 9826

About FT Works

FT Works helps technology companies create and improve their support operations. Areas of expertise include designing support offerings, creating hiring plans to recruit the right people quickly, training support staff to deliver effective support, defining and implementing support processes, selecting support tools, designing effective metrics, and support center audits. See more details at www.ftworks.com

Subscription Information

To request a subscription, please drop us a note. The mailing list is confidential and is never shared with anyone, for any reason. To unsubscribe, click here.

You may reproduce items in this newsletter as long as you clearly display this entire copyright notice. Contact us if you have questions about republications.

Back to Newsletter Archive