December 1998

Life-Cycle Testing For Call Center Quality

BY DREW KNOWLAND

With the complexity of today's integrated call center operating environment, perhaps many call center managers are resigning themselves to increased incidences of outage or serious failure. You would be hard pressed to find a call center manager who couldn't recount one or more personal experiences of such critical failures in their operations. Yet many of these outages could have been prevented by the use of automated test systems as part of a comprehensive call center quality assurance process, including in-service monitoring of call center network connections, systems, and applications.

Despite the relatively high potential for serious failure, however, many call centers take few if any preventative measures. Most call centers today perform only minimal testing of new or modified applications and systems prior to live deployment. Most testing that takes place is done manually - typically consisting of a handful of people dialing in and trying to find problems - including testing of VRUs (voice response units) and other CTI systems.

Manual testing, in most instances, simply cannot replicate the conditions that cause call center systems and applications to fail. This is because many network, system, or application failures only show up under load (normal operating conditions) or under stress (extreme operating conditions). Manual testing cannot replicate the call volumes or the variations in call type and duration that cause systems and applications to fail. Manual testing also suffers from a lack of consistency and repeatability, two critical requirements for effective test plans. As a result, it is the customers who are finding most of the bugs and problems in call centers today.

ACHIEVING IMPROVED CALL CENTER RELIABILITY
This doesn't have to be the case. Automated telecommunications test systems designed for testing CTI applications make it possible for call centers to thoroughly test their systems and applications before, during, and after deployment. Automated tools also exist for load testing and monitoring corporate Web sites. Using automated test systems and a process of life-cycle testing, call centers can attain higher levels of quality and reliability by pinpointing and eliminating system and application problems before they impact customer service.

The benefits resulting from improved application and system reliability in call centers can be significant. In a call center receiving 1,000 calls an hour, a 30-minute outage can mean 500 customers who are negatively impacted. If your Web site is down or overloaded and difficult to reach, you could be negatively impacting similar numbers of customers and you might not even know it. Even a brief outage can translate into a significant amount of lost revenue: It's simply too easy today for the customer to contact your competition. In addition to measuring the cost of lost customer transactions, you might also consider the cost of losing a customer completely. How much does it cost to find a new customer to replace one that has been lost?

QUALITY ASSURANCE IS MORE THAN MONITORING
The quality assurance process in most call centers needs to be expanded. The customer's quality experience doesn't begin when they reach an agent or customer service representative. It begins from the time they pick up their phone to call your company or attempt to access your company's Web site. A small but growing number of leading-edge call centers are employing a broader definition of quality assurance in the call center by implementing life-cycle testing programs for their call center applications.

Life-cycle testing implements a comprehensive program of testing, including:

• Feature/Function testing of new applications.
• Load & stress testing of the integrated call center components.
• Regression testing of software revisions prior to going "live."
• In-service monitoring of the call center network, systems, and applications.

Some of the measurements that a comprehensive testing process takes include:

• Time to answer at increasing load levels.
• Number of busy signals.
• Rings before answer.
• Timing between prompts.
• VRU/Host response times.
• Time in queue.
• Call/screen pop timings.
• Database response times.
• Prompt errors.
• Call handling errors.

Life-Cycle Call Center Testing
Life-cycle testing begins with automated feature/function testing during the application development phase of call center applications to ensure that bugs are identified as early as possible. Feature/Function testing breaks systems into functional areas and ascertains that all features and system interactions work to specification. Tests look at each state change in the system, and internal and external system actions, as well as identifying valid, invalid, and boundary conditions in the application.

Many call center system and application failures will only show up under load. Load and stress testing let call center developers know where, when, and how call center systems and Web sites will fail, and both types of testing need to be performed on the integrated call center and Web site systems. Individual systems may pass tests successfully, but this provides no assurance that they will work properly when they are integrated.

Duplicate Real-World Conditions
Ideally, a test plan duplicates expected calling and Web site usage conditions and patterns. Automated testing is capable of generating thousands of test calls or Web hits. Test systems should also be capable of duplicating the variable actions of real callers or Web site users, including stepping through IVR call paths, determining if the correct prompts are playing, and measuring timings. The test plan should identify stress bottlenecks in the systems and develop a Load Service Curve that can be used for capacity planning purposes.

Under life-cycle testing, regression tests are run on all new software releases and bug fixes prior to deployment on "live" systems. These tests are used to verify that nothing new has broken and that the application and system performance characteristics haven't been adversely affected. Once automated call center system and application tests have been created, regression tests are fast and easy to run.

In-Service Monitoring
If you are only evaluating quality by monitoring agent interactions or measuring service statistics like wait time, you are missing significant potential sources of customer dissatisfaction. It is possible that customer calls or contact attempts may not even be reaching the point where they show up as a statistic in your call center. This is why in-service monitoring is a vital component of a life-cycle testing program. In-service monitoring generates periodic test calls that duplicate the actions of real callers and use the PSTN and call center systems the way actual callers would.

In some call centers, up to eighty percent of customer interactions or calls are at least partially automated. Customer self-service is a growing trend, and customers today may even prefer IVR systems or the Web for many of their transactions. If these self-service systems fail, even partially, there can be serious operational and business ramifications. Customer service representatives can suddenly become inundated with calls; wait times can skyrocket; business can be lost.

Unfortunately, failures in automated customer self-service systems are not always immediately apparent. Carrier and network routing problems may be intermittent. A handful of VRU ports may not be working properly. The wrong prompts may be playing. Customers may be getting disconnected. At one major computer manufacturer, the VRUs were answering customer calls, taking customers to a prompt that played the message "All customer representatives are busy. Please call back later," and then disconnecting the call. The call center manager wasn't even aware there was a problem.

Automated Test Calls
Examples like this are why a growing number of leading call centers are implementing periodic automated testing programs. One major U.S. bank places automated test calls every night to over 4,000 VRU ports to ensure that each one is working properly. These test calls use prompt recognition and step through a representative call flow to ensure not only that the VRU ports are up and running but also that the correct prompts are playing and that there are not any excessive timing delays in database access. The test systems check to see that the call reaches the VRU, and that the VRU port answers the call, responds properly to DTMF tones, and transfers calls out of the VRU directly. If problems are found, the test system automatically pages a technical person to alert them to the trouble. This system has resulted in improved overall customer service levels for the bank and provided a potentially significant competitive advantage.

SUMMARY
The call center is the primary point of customer contact for many companies today, and quality of service in the call center can be a source of sustainable competitive differentiation. By implementing quality assurance programs in their call centers - including life-cycle testing of systems, applications, and corporate Web sites using automated test systems - companies can help prevent call center outages or overloads. Automated test systems can play a significant role in quality improvement by finding and pinpointing the source of network, system, and application failures that would otherwise negatively impact customer service and quality.

Drew Knowland is director of call center marketing at Hammer Technologies. Hammer's automated computer telephony test systems are widely used by CTI equipment manufacturers, carriers, and leading call centers for quality assurance testing. For more information, contact the company at 800-HAMMER-IT, or visit their Web site at www.hammer.com.