|
Targeting vendors, carriers, and service providers, the Performer suite is a next-generation network performance testing solution. Used on new VoIP devices and applications, it generates realistic stress levels of calls and tests quality measurements of these calls. The hardware and software components that make up the Performer suite are MediaPro, QPro, H.323Sim, SIPSim, and NetSim. MediaPro is a real-time voice over data monitor that analyzes media and signaling data generated from the H.323/MGCP/SIP protocols. QPro is a voice-quality evaluation tool that employs the Perceptual Analysis Measurement System (PAMS) to analyze the Mean Opinion Score (MOS) on circuit-switched interfaces. H.323Sim is an H.323 call generator that can emulate more than 2000 calls simultaneously at a rate of over 80,000 calls per hour. SIPSim is capable of emulating several SIP phones and implementing sessions between two proxies or IP PBX servers. NetSim simulates various WAN impairments such as latency, jitter, and packet loss to assess their effect on networks. Nearly all of these components are controlled from a single console, and can possibly automate testing through scripting.
TEST SCENARIOS
TMC Labs set up two scenarios so that we could test both an H.323 (Cisco
ATA 186 analog telephone adaptor) and a SIP product (Mediatrix 1104 device)
using the Performer suite. The setup for both scenarios were very similar
� the main difference lay in the fact that H.323Sim is completely software
oriented, whereas SIPSim is hardware-based, which allows for better testing
of the performance of SIP, according to RADCOM claims. We connected both the
Cisco ATA 186 adaptor and Mediatrix 1104 device to the QPro hardware. QPro
was linked to our network through a hub, as was the MediaPro hardware and
both VoIP devices. SIPSim hooked up to MediaPro to complete the myriad
connections. We could now make H.323 calls but only see analysis from
H.323Sim and QPro whereas the SIP calls would also draw on the services of
MediaPro. Once we set up the IP addresses for each application, we could
view the QPro, MediaPro, and SIPSim by installing the Performer Console from
any PC on our network. Although it was listed in the Performer Console and
could have been included, we did not obtain NetSim for our testing. We did
download and view H.323Sim but as a separate application.
OPERATIONAL TESTING
It was not difficult to assess many of the strengths and weaknesses of
the Performer Console when first trying out the graphical interface. When
the console first came up (Figure 1), MediaPro, QPro, and SIPSim interfaces
were brought up in a hierarchal table that indicated the status of the
applications. We could also click to any of these applications from icons on
the left side of the interface to start them. When running, we could click
on the tabs at the bottom of the interface to access each individual
application. This immediately indicated to us that the console did indeed
act as a central management GUI for most of the Performer suite (the
exception being the H.323Sim application). Although there were many screens
and options within each of the application�s interfaces, we did not find
it difficult to find the main settings we needed. However, less obvious
configuration settings may be difficult to find because of the nature of
this busy console.
It took us some time to become familiar with the console. At first, users
may be disoriented even when trying common Windows conventions because a few
of these conventions may not work with this console. For example, numbers
cannot just be typed into the correct port dialog box in QPro�s setup
screen. Instead, the Dialer, which is the button link to the channel setup,
must be used. Also, having the ability to right-click on every featured
object would be beneficial to users. Most of the right-clicking
functionality we noticed was for the layout design of the console or for
copying and pasting, not for guiding users to configuration/monitoring
settings.
We were able to perform PAMS and other measurements through the Cisco 186
adaptor from the QPro to the H.323Sim. Running these tests required a few
configurations to the H.323Sim but this was done with little difficulty. Our
test calls were running within minutes of finishing the configurations. The
PAMS scores for Cisco�s gateway averaged close to 3.5 for incoming
listening quality and slightly less than 4 for incoming listening effort.
These scores signify some degradation but are acceptable (5 would be a
perfect score). More importantly for our purposes though is that the test
consistently worked and accurate testing information could be attained.
Unfortunately, we had problems negotiating between SIPSim and the SIP
proxy server we were using, so we could not measure actual real-time SIP
calls. However, we did run tests within SIPSim itself to analyze the inner
workings of the application. In addition, we saw that MediaPro reported the
errors of calls between SIPSim and QPro, at least showing us that the SIPSim
server and MediaPro�s server were communicating with each other. While
this did not give us the full scope of how MediaPro functioned, we did
receive a demo so that we could receive a better indication of how that
application monitored the VoIP signaling data and how it worked in tandem
with the rest of the suite. Figure 3 shows MediaPro in action.
Next Version
Much of what we would have detailed as room for improvement issues will
actually be available in the next version of the Performer. These new
capabilities include a stressing mode for the QPro, which allows simulation
of a multitude of calls to stress a gateway or PSTN element while conducting
audio quality measurements, and a new capturing utility, which we had the
opportunity to look at in beta format and will be available via the
Performer Console. There will be remote management capabilities for the
Performer, such as running an IP configuration, restart, shutdown, and time
zone adjustment remotely. New MediaPro features include online analysis for
packet-loss in addition to the online analysis of jitter that is already
available, automatic instead of manual voice quality measurements, advanced
filters and sorting mechanisms, and a Find feature. A sniffer converter will
allow for the conversion of Network Associates� Sniffer trace files so
that MediaPro can analyze them. In addition to PAMS, the PESQ quality
measurement algorithm will also be available, and users will be able to
select the algorithm they would like to use. Lastly, to complete the
integration of the entire Performer suite, H.323Sim will be incorporated
into the Performer Console.
CONCLUSION
While there is other equipment and applications that may cover certain
aspects of VoIP testing better than RADCOM�s Performer, few if any of them
can match its complete functionality. The integration of all of the
applications gives any vendor or service provider the opportunity to
analyze, monitor, and measure most aspects of VoIP equipment, and it does so
using a solid, albeit not perfect, console. We recommend customers looking
for a complete VoIP testing solution to inquire more about the Performer. It
may just be what they need.
[ Return
To The March 2002 Table Of Contents ]
|
SUBSCRIPTIONS
TMCnet LOGIN
WEBINARS
