January 2002

Testing During The Deployment Of Next-Generation Networks

BY STEVE GLADSTONE

The race to deploy next-generation networks (NGN) has been quickening. 24 months ago, all activity was concentrated with the Network Equipment Manufacturers (NEMs), developing the components that would power these networks. 12 months ago, the number of evaluation and trials saw a marked increase. Now many players have begun to transition to the third phase of this process � deployment.

At the heart of this race to deploy NGNs is the promise of efficient and faster creation of higher revenue generating services (such as local number portability, and call forwarding, and unified messaging) � providing a much-needed source of competitive differentiation for service providers. At the same time, NGNs are expected to deliver substantial savings to the service providers. UBS Warburg estimates operational savings in 10 � 20 percent range and cost-savings versus legacy equipment in the 50 percent-60 percent range. The underlying value of these networks to the service provider is considerable.

However, in order to capture this value, the service provider must ensure that its NGN provides at least the same Quality of Experience (QoE) that consumers have become accustomed to with the public switched telephone network (PSTN). This is even more critical in the face of increasing competition where it is becoming easier for consumers to switch from one service provider to another.

Consequently, successful deployment of NGNs requires a new mindset in deployment testing.

QoE Defined
QoE is a new term describing the emerging reality that what ultimately matters in moving to the NGN is the experience customers have with the service.

QoE is not a specific metric, but rather a concept comprising all elements of a subscriber�s perception of the network and performance relative to expectations. More importantly, however, QoE represents a paradigm shift � a change from the testing using an engineering perspective to testing using an end user�s perspective. This implies focusing on a set of parameters that combine to determine what the end user�s experience will be. The following table describes these parameters and the level of quality that customers expect.

Creating The NGNs
The creation of the NGN can be broken into four phases, each with specific activities and desired outcomes. The service provider is a significant player in all but the first of these phases. Consequently it is important that the service provider give sufficient thought to its approach to each phase while understanding the significance each phase will have on the delivery of its NGN.

Phase I: Development
In the first phase, equipment manufactures are focused on creating the network elements (media gateways, call agents, and feature servers) for the NGNs. From a testing perspective, their needs center on validating the performance and functionality of these devices in their labs. For example, a gateway manufacturer would conduct a series of tests using load generators to determine how many simultaneous calls the gateway can support or how well do the codecs perform. Although there is still considerable activity in this phase today, 24 months ago, the activity was almost exclusively in this area.

Phase II: Evaluation
It is during this phase that the winning service providers of the NGN world are defined.

In this phase, equipment manufacturers seek to demonstrate the capabilities of their devices to service providers while the service providers themselves wish to understand if these devices will fit their needs and perform as stated in the datasheets. With the relative newness of the NGN industry, this phase has become a critical proving ground where business is either won or lost. It is during this phase where the service provider should focus heavily on QoE in choosing between one vendor�s equipment and another. To date, service providers are mainly evaluating solutions for Internet offload (or PRI offload) and tandem replacement (or toll bypass). However, as these solutions migrate towards enhanced services, the focus on QoE becomes even more important.

Because of the inherent complexity of the networks  and the amount of testing required, service providers can make considerable investments in test equipment  and custom scripting. This investment should be  leveraged by using the results from this phase as a  baseline against which to test during the next phase. Central to being able to leverage this investment is  the ability to transfer the scripts developed for tools in the lab-based trials to tools that would be used in the deployment phase.

During the evaluation of these NGNs, the service provider must adopt a holistic perspective and consider the range of usage scenarios or modes that their customers will interactive with their NGNs and ensure that these are tested. For example, NGNs are designed to handle voice, fax, and modem traffic. At present, the overwhelming majority of testing for toll bypass scenarios is focused almost exclusively on voice quality with fax and modem quality being an after thought. This is a recipe for disaster � for if the other media types are tested too late in the process and the quality is found lacking, the cost to alleviate the problem may be prohibitive. Worst yet, the problems may only be found when customers attempt to place fax or modem calls over the NGNs.

Phase III: Deployment
In the Deployment Phase, service providers are �turning up� the NGN. The key to this phase is being able to bring services online quickly with a high level of service quality. Service providers that do not make the appropriate investment in time or money in the previous phase could find themselves caught with quality or performance issues after their networks have been turned up.

Testing during the turning up process is a critical step to guaranteeing service quality. If the testing is sufficiently comprehensive and early enough in the deployment process, service providers can avoid considerable costs associated with finding bugs and sources of poor service quality post deployment. A comprehensive approach to testing during the deployment phase goes beyond the current thinking of simply using a protocol analyzer. Some forward thing service providers have begun to see the value in using a portable version of the test tools employed in the evaluation phase to run a series of test utilizing the custom scripts created in the lab. This provides the ability to definitively say what the voice, modem, and fax quality is at the point of cut over � a fundamental aspect of SLA management. The specific features and capabilities of such tools will be discussed in the next section.

The momentum is beginning to shift to this phase. The experiences that the early adopters have during this phase will set the tone for the industry and determine the eventual rate at which NGNs penetrate the mainstream.

Phase IV: Operation
Once the NGN has been successful deployed, the challenge of maintaining the NGN transfers to the operations team. It is to the advantage of the service provider to also transfer the knowledge obtained from testing in the previous phases to the operations team to aid in rapid troubleshooting and fault detection.

To facilitate a seamless transfer of knowledge across all phases of creating the NGN, it is important that the test environment be consistent. This then calls for a set of test tools that can be applied across most, if not all, of the phases.

QoE Test Tools for Deploying NGNs
At present, the de facto standard test tool used when deploying network elements is the protocol analyzer. Protocol analyzers focus primarily on signaling and provide little insight into the quality of the bearer channel � a major component of QoE. The ideal QoE test tool should replicate the behavior of end users and provide the ability to drive real voice, fax, and modem traffic through the network element being deployed. In essence, these tools should provide the ability to conduct abbreviated versions of the tests that were staged during the evaluation phase � where possible, using the same scripts. For comprehensive QoE testing, the series of tests conducted during the turn up of new network elements should include:

• Path confirmation;
• Connect latency;
• Call completion rate;
• Bearer channel quality (voice, fax, and modem quality); and
• Speech latency.

Having all of these capabilities housed in one device is highly desirable. This would not only be more economical, but would also reduce the number of tools to be carried into the field. This tool then becomes a multi-purpose device capable of not only verifying the signaling, but also the bearer quality. Part of the reason why protocol analyzers are used in isolation to deploy network elements is that, up until now, service providers had little choice for multi-purpose QoE test tools.

Key features of this QoE test tool should include:

• Lightweight and portable.
• Integrated signaling support (ISDN, CAS, and SS7)
• Simple-to-use test environment for rapid test creation, scheduling, monitoring, and reporting.
• Real voice, fax, modem, and data traffic generation.
• Protocol analysis, quality of service measurements, and SLA verification.
• Remote control for distributed network testing, POP testing, unattended remote location testing, and integration into a Network Management System.

Such as tool can also be left in the field after deployment to act as a termination point that tier 3 engineers can use to troubleshoot problems as they arise.

Final Analysis
The vast potential the next-generation network holds is unquestionable. What is in question, however, is the ability of the service provider to capture this value.

QoE testing during the deployment of NGN elements is fundamental to the service provider being able to guarantee the level of service that customers are expecting. Service providers must therefore seek out new tools that provide the ability to conduct comprehensive testing in the field and integrate the use of these tools into deployment procedures.

Those that awaken to this new paradigm will be ahead of the pack.

Steve Gladstone is the general manager of the Infrastructure Test Group of Empirix and is an acknowledged expert in testing computer telephony systems and networks. Steve founded Hammer Technologies, now a business unit of Empirix, Inc., whose products are in use today by developers of computer telephony, advanced switching, and enhanced services systems, most of the RBOCs and long-distance carriers, and at call centers of numerous Fortune 100 companies. Visit them online at www.empirix.com.

[ Return To The January 2002 Table Of Contents ]