Choosing the Right Test Method

By Peter Leppik

"You can't manage what you can't measure," and measuring how well a customer service operation meets callers' needs can be difficult. Measuring performance is a critical component of any plan to improve customer service. The wrong test method can be worse than not testing at all, since it can lead to misleading data and a false sense that everything is working well when it isn't.

The first thing to decide is what you want to measure. Everything from capacity under load to caller preferences for recorded voices can be accurately measured, but not always with the same test. Some methods produce reproducible statistics which can be used to compare different systems, while others produce only qualitative data, useful for a general sense of system quality, but not for comparison purposes.

Contacting Past Customers
Contacting past customers, or doing a follow-up survey, is a common method for quality monitoring in call centers. Either during or shortly after a call, a caller is asked if he or she wants to participate in a survey. The survey can be either automated (though an IVR system), or administered by a trained agent..

This can be a powerful method for collecting customer satisfaction data, but it has some serious drawbacks. The most important is the difficulty in getting enough caller responses to have a statistically meaningful sample. We believe 500 survey responses should be a minimum, but given the expense and difficulty in getting callers to participate, many companies stop at around 100. Statistically, you can't reliably differentiate between 75% and 95% satisfaction with only 100 calls.

Capacity (Load) Testing
Capacity testing uses an automated dialer to place hundreds or thousands of simultaneous calls to an automated system, ensuring that the system can handle the expected call volume.

Load testing is a critical part of rolling out any new application. It is best performed when the application is close to its final form, after usability and caller preference testing is complete. Making significant changes, especially in a speech recognition application, can impact the system's ability to handle the expected load.

Continued...

back to the top

 

Copyright © 2001-2003 VoiceXML Forum. All rights reserved.
The VoiceXML Forum is a program of the
IEEE Industry Standards and Technology Organization (IEEE-ISTO).