|
There is little doubt that the abundance of choice for consumers regarding voice and video services is due largely to the deployment and success of IP-based packet networks, access technology improvements, and deregulated telecom infrastructure. We are on the brink of a digital revolution in the way people communicate and consume media over fixed and mobile networks. Convergence of services has meant the Triple Play: the combined Broadband Internet, VoIP, and Digital TV are springing up from a variety of service providers as competition for the consumer heats up. There are also a variety of different consumer devices, including PCs, PDAs, and mobile phones that are now capable of receiving and sending live video images along with audio.
Consumers are aware of picture video quality, lip synching, and overall quality within audio and multimedia applications; so how will service providers assess the customers perception of the quality of voice and video applications? Rating the customer experience from a quality perspective is a new way to look at an old problem: Can service providers objectively measure and predict a customers perceived quality of experience of what they see and hear with new IP-based services like VoIP and IPTV?
Subjective Measurement
The well-known tagline, Can you hear me now? underscores the importance of voice quality and the role of subjective testing in todays service providers networks.
Service providers perform subjective testing for speech quality by selecting a panel of ordinary consumers and then subjecting each of these people to a set of voice or video samples. The consumers are then asked to evaluate the quality of voice or video calls under different testing environments and scenarios by rating each sample call in terms of a Mean Opinion Score (MOS) from 1 to 5. This subjective test data is collected and analyzed to help service providers rate the performance of real-world conditions that consumers would experience in terms of service quality. Service providers and equipment vendors continue to use subjective testing process methodologies to validate or contest their perceived service or product quality.
Subjective testing, however, is time consuming, expensive, and must be very carefully designed and executed. It also requires a specially designed testing facility.
Objective Measurement
Objective measurement methods for voice and video quality assessment represent a cost-effective alternative to subjective tests. Such methods can analyze the performance of impairments and network parameters and produce a set of MOS scores that correlate well with available subjective data without requiring a panel of human subjects. There are two main classes of objective measurement: intrusive (active) and non-intrusive (passive). Such measurements are repeatable, efficient, and fast.
Active testing techniques inject sample voice or video signals into a network so they can be captured and assessed at another point in the network allowing comparison of the degraded voice or video to the reference sample.
The world standard for intrusive assessment of end-to-end speech quality is the perceptual evaluation of speech quality (PESQ) model, ITU-T Recommendation P.862.
Non-intrusive techniques monitor live network traffic to determine the perceived quality. As a result, network capacity is not lost to testing and the service provider is assessing the actual performance experienced by customers. Benefits of non-intrusive techniques include:
allowing wider scale and denser testing;
not losing valuable network capacity for test calls;
not requiring access to end points for test signal injection;
providing measurement data directly reflecting actual customer use.
Objective Measurement Methods for VoIP
Core 3G networks and next-generation networks will be IP packet-based. Objective measurements of voice quality in VoIP (define - news -alerts) can be assessed with measurement methods that can accurately predict packet network performance from packet statistics and analyze payload to detect echo, delay, levels, and voice quality.
Simple objective measurements of network statistics, such as packet loss, delay, or jitter in a typical VoIP network provide some measurement of network QoS, but cannot alone predict consumer experience.
There are numerous examples across many technologies but, by way of example, consider a VoIP network with two percent packet loss. In the first case, the packet loss is evenly distributed in time and the packet stream is converted back to speech by an edge device with effective error correction. In this case the user experiences two percent packet loss and high voice quality. In the second case, the packet loss is bursty and the packet stream is converted back to speech by an edge device with poor error correction. In this case, the user is subject to two percent packet loss and low speech quality.
Objectively predicting MOS scores using bulk assumptions about the network and averaged performance data, such as certain methods derived from the E-Model, simply cannot reflect the perceived performance for an individual session or a real consumer experience.
Objective customer-centric quality assessment requires a complementary set of voice, audio, and video measurement methods. These cover active (known test signal) and passive (live traffic) testing requirements. Passive methods can also determine the separate performance of the IP-bearer and IP-payload. For example, in a typical VoIP network carrying a voice service, the IP-bearer measurement determines if the packet transport network service is affected. It uses a sophisticated mapping of packet and jitter temporal distribution to determine if packet loss and jitter is degrading the MOS performance of the bearer.
IP payload measurement analyzes the packet payload to measure echo, delay, speech level, noise level, and the speech quality of the payload. This analysis can be applied to a sample of the call traffic. Sampling is sufficient, since it is used to diagnose slowly varying network problems such as defective Customer Premise Equipment (levels, acoustic echo path, speech distortion, etc.), under-provisioning of echo cancellers, and quality issues.
This alternative, and more robust, VoIP monitoring measurement method provides a powerful and unique combination of IP-bearer and payload analysis. The combination of measurements, in addition to packet loss, jitter, delay, speech level, noise level, or echo, provides a comprehensive representation of customer experience and allows diagnostics and root cause analysis of service-affecting faults. This type of objective measurement provides accurate statistics that genuinely reflect customer experience.
The transient and potentially erratic nature of IP data packets means that the service providers future success will hinge on the quality of the voice or audio visual experience it delivers. And while it would be tempting to rely on conventional diagnostic measures to ensure that service levels remain high, the ultimate definition of success will depend on something rather more subjective the experience of the viewer or caller sitting in an armchair watching an IPTV stream or having a conversation on their VoIP phone. What is needed is an automated system capable of reporting back to the carrier with data that accurately reflects human perception.
Customer Experience Metric
Today, Operation Support Systems (OSS) that manage service assurance on service providers packet networks have been designed to handle data services rather than real-time video and multimedia services, where packet loss and delay have a much more disruptive effect on the end users experience.
In IPTV, VoIP, and other real-time triple play services, quality of service can only truly be measured if it relates to the end users experience. All other data becomes a string of meaningless figures unless the customer is satisfied with an excellent audio or visual experience, which is highly subjective, hinging on human perception.
Objective measurement metrics that predict customer experience are invaluable planning and monitoring tools for service providers that are concerned with the introduction and penetration of newer IP-based voice and video services in their markets. Accurate correlation to a wide range of subjective testing results provides essential validation and credibility to using these metrics.
Ultimately, the end user will not tolerate anything that negatively impacts the quality of their communications, whether voice, data, or video. Carriers need to ensure that the customers high expectations for quality are met. IT
Nav Chandler is director of product management at Psytechnics. For more information, please visit the company online at www.psytechnics.com (news - alerts).
If you are interested in purchasing reprints of this article (in either print or PDF format), please visit Reprint Management Services online at www.reprintbuyer.com or contact a representative via e-mail at [email protected] or by phone at 800-290-5460.
|
SUBSCRIPTIONS
TMCnet LOGIN
WEBINARS
