|
Voice Application And
Infrastructure Testing: A 'Win-Win' Proposition
By Charlie Grzybinski,
Empirix
Self-service applications based on IVR, CTI and voice recognition
technologies have become essential features of the successful enterprise
contact center ' and no wonder. A recent GartnerGroup study indicated that
the average cost of a single agent-based transaction ranges from $5.50 to
$7.00, while the same transaction using a self-service IVR application runs
a mere 45 cents. The fiscal logic is inescapable: over 55 percent of
respondents to a just-released survey of contact center professionals said
that the implementation of additional self-service alternatives is a high
priority for the coming year.1
The problem, of course, is that self-service applications only pay off if
customers use them. Performance problems, such as slow databases or host
responses, incorrect call routings and failed transactions will cause
customers to 'zero out' to an agent ' at first in frustration and
eventually by reflex ' taking a heavy toll in agent productivity, line
charges and customer satisfaction. Self-service applications are only as
strong (and profitable) as their weakest links and given the growing
complexity and sophistication of today's IVR- and CTI-based systems, there
are a lot of links to worry about. For far too many companies, the expected
operating savings from these self-service applications fail to materialize
as many customers opt out to agents due to performance problems. Almost
everyone has a personal horror story about being stuck in an endless IVR
loop, routed to the wrong person or repeatedly asked for the same
information. None of this means that self-service IVR and CTI applications
(and ultimately, fully Web-enabled contact center operations) aren't the
way of the future. It does mean, however, that when investing in and
operating these new technologies, contact center managers need to be sure
they are getting the most from these valuable workhorse systems.
How do contact centers ensure that they're getting what they pay for in
self-service applications? The answer is both straightforward and
cost-effective: comprehensive, automated testing of IVR/CTI applications.
This preventive measure is the key to making certain that self-service
applications live up to their full potential in contact center environments.
Automated testing in the design and predeployment stages of application
development ensures that contact center managers pinpoint and fix
scalability issues and performance bottlenecks before an application goes
'live' (and contact centers end up testing their customers' patience
and loyalty as they try to fix problems in production).
There are now product- and service-based solutions on the market that
enable contact center operation managers, application developers, QA
engineers and technical staff to pinpoint and resolve performance problems
before they undermine customer confidence and drive up operating costs.
These automated testing solutions reveal hitherto expensive and hard-to-find
scalability problems by emulating thousands of callers in real-world traffic
patterns that accurately assess the performance of an application and its
underlying voice infrastructure ' before and after the application is
deployed.
Quality By The Numbers
Automated, predeployment load and performance testing helps contact
centers fulfill multiple operating objectives. First and foremost, it takes
a comprehensive approach to performance from the customer's perspective:
rather than providing a narrow view of whether each subsystem is working, it
looks at an entire transaction from the PSTN all the way out to an agent. By
measuring performance at each step of the test script under real-world
conditions, automated testing finds application errors and performance
bottlenecks by tracking the efficiency and interoperability of speech
recognition servers, IVRs, database servers, middleware components, WAN/LAN
network connections, switches, routers and ACD/load balancers.
Second, automated predeployment testing provides clear and detailed
performance reports that help contact centers significantly improve the
performance of their systems to ensure that vendors and service providers
live up to their claims and that customers receive optimal quality of
experience (QoE).
Pre-deployment testing is a fast, cost-effective way to boost performance
and quality, ensuring that customers are happier about using self-service
applications and that hidden operating costs are eliminated before an
application 'goes live.'
Third, good testing delivers a firm foundation of archived data (and
reusable test scripts) that provide a basis for corrective retesting,
performance benchmarking and ongoing monitoring of self-service
applications.
In short, predeployment testing turns out to be the most cost-effective
way to ensure that voice applications and infrastructure will scale to meet
performance and business objectives, thereby enabling contact centers to
realize the full potential of their investment in self-service applications.
Four Steps To Effective Testing
The most effective and comprehensive testing processes generally follow
a four-step protocol. The first step is to set the test objectives and
design the testing methodology. The crucial requirements for this step are
to understand the business requirements of the system (e.g., to handle up to
10,000 calls per hour) and the metrics to be used in measuring system
performance. Detailed performance metrics should be developed for each of
the components (e.g., 99 percent of all database response times are below
three seconds) and these component metrics should add up to the overall
performance metrics and an acceptable QoE for your customers.
The next step is to establish a project timeline and assemble an
effective test team that includes, at a minimum, senior testers, database
administrators, telephony architects and the application developers
themselves. In defining both the timetable and the team's work
assignments, managers should build adequate retest time into the project
plan, since newly developed or updated IVR and CTI systems invariably
manifest more problems than expected.
Once a project plan is in place and performance metrics are agreed upon,
the test plan should be mapped against the system architecture to ensure
that all mission-critical transactions and components are stressed during
the test. To ensure that the system will continue to scale as the business
grows, each high-volume or high-value call path should be exercised at a
level of intensity based on realistic projections of future capacity needs.
The fourth step in the process is to execute the test to identify errors
and performance bottlenecks. A typical test will consist of many iterative
testing sessions in which the test is run, problems are found and the test
is stopped so the problems can be fixed before testing resumes. Prior
agreement on performance metrics and methodology will enable the test team
to quickly work through the issues and fine- tune performance.
Once every component of the system is stressed and fine-tuned and
follow-up tests indicate that users will enjoy adequate QoE, all performance
issues and improvements should be documented to facilitate problem-solving
in future tests or in a production environment.
Common Obstacles To Effective Testing
Unfortunately, a host of internal obstacles can impede ' and sometimes
prevent ' effective testing of new or existing applications and
infrastructure. Many companies lack the tools, budget or staff to conduct
testing or often are under such tight deadlines that the testing process is
curtailed (or even eliminated entirely). Other companies may have the tools,
but lack technicians who know how to use them. Even those companies that
possess both the tools and technical talent often lack the expertise needed
to design tests that can effectively stress the system and find performance
issues.
Perhaps the most daunting obstacle is the sheer organizational complexity
of today's contact center environment. Different vendors supply different
parts of the system ' parts that are often cobbled together by a third
party. Key components (IVR, database, switch, etc.) may be geographically
dispersed and under the control of different parts of the organization, or
even outsourced. Potential problems can occur in any component ' the
switch, IVR, CTI, back-end databases, etc. ' or with the network
connecting the components.
Of course, simply recognizing that these obstacles exist ' and
addressing them in a comprehensive plan ' can help a contact center
minimize their impact on system quality and performance. In addition, test
managers should make every effort to ensure not only that everyone agrees in
advance on objective performance standards, but also that test data are
distributed as widely as possible, so all interested parties are working
toward the same goals.
Managed Testing Services Can Verify Scalability Quickly And Affordably
One of the easiest, quickest and most affordable ways to overcome these
internal hurdles is to enlist a qualified third-party to test new or
existing applications and infrastructure. Because call volumes can be
generated over the PSTN, the setup process is relatively fast and simple.
Often, companies testing internally check only for average volumes on an
average day, which often causes testers to miss many common but expensive
problems. Another common pitfall is to test only for those circumstances in
which callers do everything correctly rather than conduct 'negative
testing' ' or testing miscues by callers. Third-party testing experts
typically design many different scenarios to find problems that may only
occur under a certain set of traffic conditions, after a certain amount of
run time, at particular times of day or under different types of load.
Testing companies can also provide a suite of tools that allow contact
centers to access and analyze performance data in real time ' helping
everyone, regardless of geographic location, get on the same page and
examine performance of all aspects of the application and infrastructure
(switch, database, etc.). Effective testing can speed the application
development process and reassure companies that their application will
scale. Testing also provides the necessary performance data to verify that
vendors and service providers are living up to service agreements or ROI
promises. Testing companies can also document performance improvements and
store these test scripts and results online, providing easier access to
interested parties throughout the entire development and production life.
Getting What You Pay For
Whether performed by outside experts or by a properly trained and
equipped internal team, automated testing of IVR, CTI and voice recognition
applications is an indispensable step in ensuring that these important
contact center technologies live up to their full potential. Without a
sensible testing plan, self-service applications can actually exacerbate the
very customer service and operating cost problems they are supposed to
solve. With the rich and detailed data provided by automated, end-to-end
transaction testing, contact center managers can enhance their customers'
QoE, free up agent resources and take charge of their toll and hardware
costs. For contact centers and callers alike, automated testing is a 'win-win'
proposition.
Charlie Grzybinski is senior voice consultant at Empirix (www.empirix.com), a test and monitoring
company for voice, Web and network applications. He has over 13 years'
experience consulting to Fortune 1000 customers regarding test strategy and
execution for Web, voice and network applications and infrastructure.
1 Benchmark Portal, April 2002.
[ Return
To The August 2002 Table Of Contents ]
|
