[July 26, 2002]

You've Got The Right Metrics... Now What?

BY BOB MASSAD

IP is in the call center. And now, voice over IP (VoIP) is being deployed at an increasing pace in the call center to take advantage of the many benefits of packet-based networks. For example, traditional call centers were built upon proprietary products from "specialist" vendors, but VoIP-based centers are built on open standards that allow for a wider range of product choices at lower costs without "lock in" to a particular vendor.

From an application perspective, a key driver of IP and VoIP in the call center has been emergence of the Web-enabled call center. Here, after online customers have found what they are looking for, they can immediately "click to talk" to a call center agent. VoIP significantly enhances this application and the transaction experience as the call can be made over the Internet without having to tear down the Web session. Ultimately, the use of VoIP leads to improved business, productivity, and customer satisfaction levels. But what if the call quality of the VoIP call is not acceptable, degrades the experience, and interferes with the business transaction?

As a network operations person responsible for the effective operation of the network tools provided for call center agents, such as VoIP services, how would you know that call quality was an issue? How would you know ahead of time? How would you prevent it? How would you find and fix the problem? These questions are what this article addresses.

Fundamentals
In a previous article ("VoIP In The Call Center: A Guide To Call Quality And SLA Metrics"), we addressed the topic of making VoIP successful in the call center by noting that the first step was to understand how to measure the quality and effectiveness of the VoIP implementation. We pointed out that VoIP, and for that matter emerging video over IP applications, are not of a similar nature to the "data" network applications we have become accustomed to. VoIP and video are real-time, isochronous, sensory or experiential applications that use UDP (User Datagram Protocol) as the transport.

In contrast, the prevalent applications running over the Internet today, such as file transfer, Web access, and e-mail, are non-real-time, asynchronous and use TCP (Transport Control Protocol) for transport. VoIP is real time and isochronous because it is an experiential application where end-user perception is what really matters. It uses UDP because it cannot afford the complexities and delays presented by TCP. At a finer level, TCP is a connection-oriented, acknowledgement based, flow controlled, reliable, and multiplexed service running on top of IP. UDP is really just a multiplexing service. No connection. No acknowledgement. No flow control. It just blasts packets.

These differentiating aspects of VoIP (when compared with traditional data applications) require a new management paradigm. Management and management data must adapt to the fundamental nature of IP packet-based networks, since it is that fundamental nature -- i.e., that packet networks are bursty and heterogeneous environments, and the lack of TCP-like controls -- that presents problems to "toll quality" VoIP. The value of those TCP controls is generally considered not worth their cost or the trade-off against delay. Minimizing delay in a real-time application is considered more important than the acknowledgement, retransmission, congestion back-off services, and the accompanying high overhead that TCP provides.

This new management paradigm for VoIP application and transport service management requires a focus on burstiness, since there is no flow control offered by UDP; a focus on end-user perception, since it is a sensory or experiential app; and a focus on packet-loss, since UDP is an unreliable service. Putting these together, a management model for VoIP would primarily seek to measure burst packet loss and its affect on end-user perception.

A Word About Jitter
As a quick aside, jitter is also a problem derived from the heterogeneity of IP networks. Many different sized packets, support for a variety of applications, queue overflows in routers, and the use of a shared network cause irregularities in inter-packet arrival times. To compensate, VoIP endpoints implement a jitter buffer, which adds minimum delay in order to create a constant playout of packets at the receiver. When jitter is excessive, causing packets to arrive outside the jitter buffer delay window, then jitter buffer discards occur. We treat these discards as packet loss. However, being able to distinguish between packet loss from the network and packet loss or discards from the jitter buffer is crucial in managing the VoIP network.

Troubleshooting Tools
In our previous article, we said that a rigorous management approach would implement call quality monitoring agents into endpoints such as media gateways and IP phones as well as protocol/network analyzers and RMON probes. This approach allowed for better problem isolation, problem confirmation, and the automatic spawning of more detailed troubleshooting activities such as alarm triggering or a packet "capture and decode." New activities taking place within standards-setting bodies such as IETF are discussing protocol extensions to allow endpoint and management devices, sometimes called midpoints, to communicate voice quality information which allows for better and regular "tuning."

We also previously noted that traditional measurement techniques employed by analyzers and probes were ineffective in that they had no inherent concept of codec type, burstiness, end-user perception, or even "quality," and that they required a tremendous amount of data, time, and correlation work by the user in order to get a sense of call quality and its contributing factors.

Two Scenarios
For example, traditional management devices such as analyzers and probes provide discrete rather than synthesized statistics. To test the utility of this approach, can you tell if a caller on a call with 3% packet loss, 10ms jitter, 100ms delay, and using G.711 experienced good call quality or poor call quality? Is it better or worse than a call with 4% packet loss, 5ms jitter, 80ms delay, using G.729?

Thus, traditional methods may actually obscure the issue by providing data that is:

1. not directly pertinent,
2. averaged,
3. misleading.

A traditional probe may tell you that packet loss was x% for a call. But you don't know if the packet loss was evenly spaced, and thus "concealed" by the codec, or occurred in bursts, nor whether the loss number reflects losses just at the media "interface" or includes packets discarded by the jitter buffer, each of which would be caused for different reasons, leading to different correction measures. The typical analyzer or probe has no insight into jitter buffer activity even though it is quite possible to have very few packets lost at the interface and many more discarded, i.e. lost, by the jitter buffer, thus severely understating the real or effective packet loss count.

Analyzing The Data
Knowing now how to monitor and measure VoIP call quality, the next step is to see how to analyze the data. To analyze call quality measurements, the tool must be able to break VoIP quality degradation down into its main constituent parts and then assess or understand each parts contribution to call quality. Only then can specific positive action be taken to improve performance. Recall the above sample scenarios.

The key components influencing VoIP call-quality are codec type, packet loss, jitter, delay (primarily with respect to conversational call quality) and end-user perception. Packet loss data can further break down between burst and random or isolated packet loss information. Once the proportional impact of each component is understood, the network manager can determine which components need to be better accommodated or adjusted.

For example, if the network implements G.723-compliant codecs and the codec type is the biggest contributor to poor call quality, usually measured as a R-factor or MOS, the codec configuration can be changed to G.729 or G. 711 to improve call quality, though at the cost of higher bandwidth consumption. If jitter emerges as the biggest contributing factor to poor quality -- a fact which you can not deduce by simply looking at the measured jitter level but only by measuring the effects of jitter -- then the jitter buffer delay window can be extended, or a switch from a static to an adaptive jitter buffer configuration can be made, among other options. If the tools used are not capable of these types of measurements and drill downs, substantial resources will be expended trying to get to the source of the problem.

Call Quality
I mentioned conversational call quality. It is very important to understand that a meter or monitor must be able to distinguish between and rate both listener call quality and conversational call quality. Listener call quality or call clarity is virtually unaffected by delay. But conversational quality can be devastated by high periods of delay. You can imagine a live broadcast -- one speaker but N listeners -- experiencing a delay of 250ms as the only impairment. The listener may become mildly annoyed but will still be able to understand the broadcast. Using the same delay time in a conversation, speakers will invariably "talk over" each other and there will be little or no understanding of what was said -- a very poor call.

To summarize, analyzing VoIP call quality is very dependent on knowing what really affects VoIP call quality, given that it is a fundamentally different application than the applications prevalent on our networks today, and so must be managed with a different focus than today's prevalent application. Troubleshooting tools must be tuned to observe, collect, and synthesize the relevant data in real time such that an overall single number quality score can be obtained in real time. Once obtained, along with the sense that a call was good or bad, the degradation in a bad or imperfect call can be broken down into it's major contributing factors, each of which may point to a different corrective action strategy in real time. VoIP is a real-time application. With regard to quality, your opinion counts.

Bob Massad is VP of marketing for Telchemy. For more information, please visit their Web site at www.telchemy.com.