November 2000

As the Internet and convergence penetrate of our daily lives, the buzz of applications increases: e-commerce, wireless connectivity, personalization of everything from marketing pieces to Web sites to HTTP 404 errors, convergence of things once thought un-convergable. For most people, convergence means the usable layer -- the interfaces and applications that they deal with everyday. But for a few people behind the scenes, convergence means something else. Put bluntly, it means a testing nightmare.

Testing used to be so simple. I remember in high school shop when I first used an oscilloscope. Up to that point I had used -- okay, I admit it... I owned! -- an ohmmeter/voltmeter combo and a few other testing accessories. But I had never imagined what a 'scope might be able to tell you about the goings-on inside of a circuit.

Well, the days when you only needed a 'scope to troubleshoot are long gone. In fact, specific applications have always required specific testing equipment, but a generalist like me could get by with a few tools like a 'scope and an ohmmeter. Enter convergence. Enter the call center-enterprise-e-commerce-DIY-online-auction-all you-can-eat super bank-market-site-e-mail-service-provider thing (aka, the dot-com). And enter their network. Now, try to test it.

Testing a converged network of any scale requires a wealth of specialty knowledge well beyond what might be reasonably expected of any individual testing person. Maybe even beyond the pale of a decent MIS department. So what is the poor business person to do? His salespeople want to access their e-mail from the road. Her agents must handle Web-based IVR, e-mail, VoIP chat, and traditional phone calls transparently. But somehow things aren't working. And figuring out why seems tough -- even tougher than programming a VCR, or figuring out how to turn off that annoying pop-up clipboard in Word 2000.

Well, at the heart of all of this are two things: convergence, and testing to insure convergence is actually taking place. To paraphrase Shakespeare: "Convergence is not convergence which alters when it another converged application finds."

TOWARD A TESTING STRATEGY
As I said, testing the converged network requires knowledge. In a way, that's exactly what testing products are. It's like you're buying a hardened, productized form of knowledge. Knowledge of your network that can be passed from user to user, that doesn't leave when another company offers it better stock options.

A while ago, we ran some articles on testing that discussed the need for businesses of all sorts to adopt more holistic policies involving real-life and life cycle testing. Now, to get a feel for how users put together their testing strategies, I asked a few people in the industry whether these earlier cries have fallen on deaf ears. The response was encouraging.

Brian Miller, manager of the contact center testing and monitoring group with Empirix, had this to say: "There has been high interest in automated quality monitoring solutions that are testing the system in the same way that a customer would use it. Traditional monitoring solutions won't necessarily identify performance problems that directly affect the customer experience (back end slow downs, incorrect or garbled voice prompts, etc.)."

As CTI technologies become more of a "must have" than a "maybe," end-to-end testing is increasingly required. This is not to say that end-to-end testing is easy to achieve, or even fully possible in every situation. As Brian Mason, VoIP product manager for SmartBits Product Line of Spirent Communications, points out, there are two main hurdles to testing in this manner:

"The first hurdle is that the complexity of large networks today makes it hard to test the entire network from end-to-end. Once a problem is identified it is still hard to isolate the problem and to figure out how to resolve it... And no single piece of test equipment today can test an entire network. The second problem with testing a network from end-to-end under real-live load conditions is tying up network resources. Whereas, to test an individual component you may only have to tie up that single device, and if you take it out of the network you do not use up any network bandwidth. So while component type testing is not always the most realistic, it is often the easiest and unfortunately what many companies end up doing. A good compromise is to create as realistic a test lab as possible that allows you to simulate real life situations."

24x7 SERVICE
ASPs are all the rage these days, and understandably so. What they offer is the implementation of a basic business rule -- focus on your core competencies. They do this by saying, "Let us handle your network. We know it has to work all the time; we know it has to provide reliable connectivity, bandwidth, and enhanced features. We'll handle it for you. You just go about your business."

So, are different strategies employed when testing for enterprise/call center type environments versus testing service provider environments? What particulars, if any, might be of more interest to a service provider than an enterprise or call center?

Brian Miller commented on the place of economics and technology in testing: "A financial institution receiving five million calls per month will be concerned about dropped calls because some percent of those calls will be lost transactions or potential lost customers. This will drive these companies to focus on testing solutions that can continuously monitor the customer experience and then identify key failures in service level. The test investment will focus on monitoring the automated applications. On the other hand, a service provider or an enterprise that has most of the calls going to an agent will focus much more on technology that allows automated monitoring of performance through to the agent.

"Technology can also drive test strategy. Enterprises are accelerating deployment of Web-enabled call centers that allow customers to interact with agents through a variety of channels. This in itself isn't new. What is new is the ability to opt out of one medium into another and to integrate these channels into a common set of workflow rules to route customer inquiries. Because all of these channels share a common database and common workflow processing, it is important to test their interaction. This requires coordination of virtual Web, voice, and e-mail users, including the ability to ramp the load on any one of these channels and measure the resulting performance impact."

Alan Sguigna, VP of marketing for Adtech Product Line, Spirent Communications, had some comments more relating to protocols: "Testing strategies for enterprise/call center applications versus service provider environments are significantly different, particularly in the areas of protocol support and feature test. For example, VoIP signaling in the enterprise space is typically based upon the ITU's H.323 protocol, whereas the carriers are deploying protocols such as MGCP, Megaco/H.248, or SIP. This is because service providers are looking for simple protocols to support basic Class 4 and Class 5 services, whereas the enterprises are looking for the rich multimedia support of H.323.

"The same distinction applies on the voice side of the VoIP equation. Enterprises are looking for analog/digital phone set support, with basic CAS and PRI signaling. Carriers are looking for SS7, GR.303, and other such interfaces. In both cases they are looking for a robust underlying IP or ATM infrastructure which can provide reliable voice QoS."

And Van Do, VP of marketing for Zarak Product Line, Spirent Communications, also had some comments: "Generally, enterprise networks, call centers, and service providers require the following common testing features: check PSTN connectivity, connectivity to IP networks, and performance under heavy load; emulation of incoming calls; and termination of outgoing calls.

"The differences in testing strategies involve the intensity of testing software applications. Enterprise networks and call centers usually require more functional testing, which is the only way for environments like call centers to ensure appropriate operation of the software applications behind the scenes. Scripting from within the test call becomes vital to performance assessment. In addition to the number of calls serviced, there is concern with reaction time, correct prompts, and correct processing of information. The service provider concerns itself more with the quality of service, the number of calls it can connect, and billing. Service providers apply the most intense of tests at the integration stage of equipment and software. After integration, maintenance tests are performed at a smaller capacity but at regular frequencies."

WIRELESS & TESTING
And what about wireless? Sure, once the traffic hits the base station, it's should be the same as any other PSTN traffic, right? But what about when using an IVR? Could something happen in the air that might interfere with service? I've occasionally had voice mail appear on my own phone a day or so late. How might wireless affect enterprise/call center/service provider needs, and what might be its special testing requirements?

Brian Miller was reassuring on this point: "To date, we have not seen a huge effect on testing. Over time, as use of voice-based browsing, unified messaging systems, and other applications that use advanced speech recognition to convert voice inputs into digital becomes more prevalent, verifying the functionality of these systems under a variety of user conditions, including wireless callers, will become increasingly important."

Van Do's comments were also reassuring: "When testing a wireless interface, more emphasis is placed on testing the connection for quality and ability to pass modem traffic. Other than that, the call is treated as a wired call. There are too many variations of physical wireless interface to be able to provide standard testing tools. However, equipment manufacturers are usually clever enough to design in wired interfaces to the wireless protocols, allowing test labs to verify the accessibility of the network."

MORE THAN JUST A TEST
Developing a testing strategy depends heavily on your business needs and how much or how little of your communications you decide to handle in-house. But once in a place, an effective testing strategy helps keep you up and running with minimal intrusion and, most importantly, minimal surprises.

[ Return To The November 2000 Table Of Contents ]