Testing During The Deployment Of
BY STEVE GLADSTONE
The race to deploy next-generation networks (NGN) has been quickening. 24
months ago, all activity was concentrated with the Network Equipment
Manufacturers (NEMs), developing the components that would power these
networks. 12 months ago, the number of evaluation and trials saw a marked
increase. Now many players have begun to transition to the third phase of
this process ï¿½ deployment.
At the heart of this race to deploy NGNs is the promise of efficient and
faster creation of higher revenue generating services (such as local number
portability, and call forwarding, and unified messaging) ï¿½ providing a
much-needed source of competitive differentiation for service providers. At
the same time, NGNs are expected to deliver substantial savings to the
service providers. UBS Warburg estimates operational savings in 10 ï¿½ 20
percent range and cost-savings versus legacy equipment in the 50 percent-60
percent range. The underlying value of these networks to the service
provider is considerable.
However, in order to capture this value, the service provider must ensure
that its NGN provides at least the same Quality of Experience (QoE) that
consumers have become accustomed to with the public switched telephone
network (PSTN). This is even more critical in the face of increasing
competition where it is becoming easier for consumers to switch from one
service provider to another.
Consequently, successful deployment of NGNs requires a new mindset in
QoE is a new term describing the emerging reality that what ultimately
matters in moving to the NGN is the experience customers have with the
QoE is not a specific metric, but rather a concept comprising all
elements of a subscriberï¿½s perception of the network and performance
relative to expectations. More importantly, however, QoE represents a
paradigm shift ï¿½ a change from the testing using an engineering
perspective to testing using an end userï¿½s perspective. This implies
focusing on a set of parameters that combine to determine what the end
userï¿½s experience will be. The following table describes these parameters
and the level of quality that customers expect.
Creating The NGNs
The creation of the NGN can be broken into four phases, each with
specific activities and desired outcomes. The service provider is a
significant player in all but the first of these phases. Consequently it is
important that the service provider give sufficient thought to its approach
to each phase while understanding the significance each phase will have on
the delivery of its NGN.
Phase I: Development
In the first phase, equipment manufactures are focused on creating the
network elements (media gateways, call agents, and feature servers) for the
NGNs. From a testing perspective, their needs center on validating the
performance and functionality of these devices in their labs. For example, a
gateway manufacturer would conduct a series of tests using load generators
to determine how many simultaneous calls the gateway can support or how well
do the codecs perform. Although there is still considerable activity in this
phase today, 24 months ago, the activity was almost exclusively in this
Phase II: Evaluation
It is during this phase that the winning service providers of the NGN
world are defined.
In this phase, equipment manufacturers seek to demonstrate the
capabilities of their devices to service providers while the service
providers themselves wish to understand if these devices will fit their
needs and perform as stated in the datasheets. With the relative newness of
the NGN industry, this phase has become a critical proving ground where
business is either won or lost. It is during this phase where the service
provider should focus heavily on QoE in choosing between one vendorï¿½s
equipment and another. To date, service providers are mainly evaluating
solutions for Internet offload (or PRI offload) and tandem replacement (or
toll bypass). However, as these solutions migrate towards enhanced services,
the focus on QoE becomes even more important.
Because of the inherent complexity of the networks and the amount
of testing required, service providers can make considerable investments in
test equipment and custom scripting. This investment should be
leveraged by using the results from this phase as a baseline against
which to test during the next phase. Central to being able to leverage this
investment is the ability to transfer the scripts developed for tools
in the lab-based trials to tools that would be used in the deployment phase.
During the evaluation of these NGNs, the service provider must adopt a
holistic perspective and consider the range of usage scenarios or modes that
their customers will interactive with their NGNs and ensure that these are
tested. For example, NGNs are designed to handle voice, fax, and modem
traffic. At present, the overwhelming majority of testing for toll bypass
scenarios is focused almost exclusively on voice quality with fax and modem
quality being an after thought. This is a recipe for disaster ï¿½ for if the
other media types are tested too late in the process and the quality is
found lacking, the cost to alleviate the problem may be prohibitive. Worst
yet, the problems may only be found when customers attempt to place fax or
modem calls over the NGNs.
Phase III: Deployment
In the Deployment Phase, service providers are ï¿½turning upï¿½ the NGN.
The key to this phase is being able to bring services online quickly with a
high level of service quality. Service providers that do not make the
appropriate investment in time or money in the previous phase could find
themselves caught with quality or performance issues after their networks
have been turned up.
Testing during the turning up process is a critical step to guaranteeing
service quality. If the testing is sufficiently comprehensive and early
enough in the deployment process, service providers can avoid considerable
costs associated with finding bugs and sources of poor service quality post
deployment. A comprehensive approach to testing during the deployment phase
goes beyond the current thinking of simply using a protocol analyzer. Some
forward thing service providers have begun to see the value in using a
portable version of the test tools employed in the evaluation phase to run a
series of test utilizing the custom scripts created in the lab. This
provides the ability to definitively say what the voice, modem, and fax
quality is at the point of cut over ï¿½ a fundamental aspect of SLA
management. The specific features and capabilities of such tools will be
discussed in the next section.
The momentum is beginning to shift to this phase. The experiences that
the early adopters have during this phase will set the tone for the industry
and determine the eventual rate at which NGNs penetrate the mainstream.
Phase IV: Operation
Once the NGN has been successful deployed, the challenge of maintaining
the NGN transfers to the operations team. It is to the advantage of the
service provider to also transfer the knowledge obtained from testing in the
previous phases to the operations team to aid in rapid troubleshooting and
To facilitate a seamless transfer of knowledge across all phases of
creating the NGN, it is important that the test environment be consistent.
This then calls for a set of test tools that can be applied across most, if
not all, of the phases.
QoE Test Tools for Deploying NGNs
At present, the de facto standard test tool used when deploying network
elements is the protocol analyzer. Protocol analyzers focus primarily on
signaling and provide little insight into the quality of the bearer channel
ï¿½ a major component of QoE. The ideal QoE test tool should replicate the
behavior of end users and provide the ability to drive real voice, fax, and
modem traffic through the network element being deployed. In essence, these
tools should provide the ability to conduct abbreviated versions of the
tests that were staged during the evaluation phase ï¿½ where possible, using
the same scripts. For comprehensive QoE testing, the series of tests
conducted during the turn up of new network elements should include:
- Path confirmation;
- Connect latency;
- Call completion rate;
- Bearer channel quality (voice, fax, and modem quality); and
- Speech latency.
Having all of these capabilities housed in one device is highly
desirable. This would not only be more economical, but would also reduce the
number of tools to be carried into the field. This tool then becomes a
multi-purpose device capable of not only verifying the signaling, but also
the bearer quality. Part of the reason why protocol analyzers are used in
isolation to deploy network elements is that, up until now, service
providers had little choice for multi-purpose QoE test tools.
Key features of this QoE test tool should include:
- Lightweight and portable.
- Integrated signaling support (ISDN, CAS, and SS7)
- Simple-to-use test environment for rapid test creation, scheduling,
monitoring, and reporting.
- Real voice, fax, modem, and data traffic generation.
- Protocol analysis, quality of service measurements, and SLA
- Remote control for distributed network testing, POP testing,
unattended remote location testing, and integration into a Network
Such as tool can also be left in the field after deployment to act as a
termination point that tier 3 engineers can use to troubleshoot problems as
The vast potential the next-generation network holds is unquestionable.
What is in question, however, is the ability of the service provider to
capture this value.
QoE testing during the deployment of NGN elements is fundamental to the
service provider being able to guarantee the level of service that customers
are expecting. Service providers must therefore seek out new tools that
provide the ability to conduct comprehensive testing in the field and
integrate the use of these tools into deployment procedures.
Those that awaken to this new paradigm will be ahead of the pack.
Steve Gladstone is the general manager of the Infrastructure Test
Group of Empirix and is an acknowledged expert in testing computer telephony
systems and networks. Steve founded Hammer Technologies, now a business unit
of Empirix, Inc., whose products are in use today by developers of computer
telephony, advanced switching, and enhanced services systems, most of the
RBOCs and long-distance carriers, and at call centers of numerous Fortune
100 companies. Visit them online at www.empirix.com.
To The January 2002 Table Of Contents ]