ITEXPO begins in:   New Coverage :  Asterisk  |  Fax Software  |  SIP Phones  |  Small Cells

Product Reviews
 March 2002

Performer Version 1.5

6 Forest Avenue
Paramus, NJ 07652
Phone: 201-518-0033
Fax: 201-556-9030
Web site: http://www.radcom-inc.com

Price: Starts at $15K and is based on configuration -- users can decide on the exact configuration required for their testing needs.

Editor's Choice Award

Installation: 4.5
Documentation: 4.75
Features: 4.5
GUI: 4.5
Overall: A-

Targeting vendors, carriers, and service providers, the Performer suite is a next-generation network performance testing solution. Used on new VoIP devices and applications, it generates realistic stress levels of calls and tests quality measurements of these calls. The hardware and software components that make up the Performer suite are MediaPro, QPro, H.323Sim, SIPSim, and NetSim. MediaPro is a real-time voice over data monitor that analyzes media and signaling data generated from the H.323/MGCP/SIP protocols. QPro is a voice-quality evaluation tool that employs the Perceptual Analysis Measurement System (PAMS) to analyze the Mean Opinion Score (MOS) on circuit-switched interfaces. H.323Sim is an H.323 call generator that can emulate more than 2000 calls simultaneously at a rate of over 80,000 calls per hour. SIPSim is capable of emulating several SIP phones and implementing sessions between two proxies or IP PBX servers. NetSim simulates various WAN impairments such as latency, jitter, and packet loss to assess their effect on networks. Nearly all of these components are controlled from a single console, and can possibly automate testing through scripting. 

TMC Labs set up two scenarios so that we could test both an H.323 (Cisco ATA 186 analog telephone adaptor) and a SIP product (Mediatrix 1104 device) using the Performer suite. The setup for both scenarios were very similar � the main difference lay in the fact that H.323Sim is completely software oriented, whereas SIPSim is hardware-based, which allows for better testing of the performance of SIP, according to RADCOM claims. We connected both the Cisco ATA 186 adaptor and Mediatrix 1104 device to the QPro hardware. QPro was linked to our network through a hub, as was the MediaPro hardware and both VoIP devices. SIPSim hooked up to MediaPro to complete the myriad connections. We could now make H.323 calls but only see analysis from H.323Sim and QPro whereas the SIP calls would also draw on the services of MediaPro. Once we set up the IP addresses for each application, we could view the QPro, MediaPro, and SIPSim by installing the Performer Console from any PC on our network. Although it was listed in the Performer Console and could have been included, we did not obtain NetSim for our testing. We did download and view H.323Sim but as a separate application.

It was not difficult to assess many of the strengths and weaknesses of the Performer Console when first trying out the graphical interface. When the console first came up (Figure 1), MediaPro, QPro, and SIPSim interfaces were brought up in a hierarchal table that indicated the status of the applications. We could also click to any of these applications from icons on the left side of the interface to start them. When running, we could click on the tabs at the bottom of the interface to access each individual application. This immediately indicated to us that the console did indeed act as a central management GUI for most of the Performer suite (the exception being the H.323Sim application). Although there were many screens and options within each of the application�s interfaces, we did not find it difficult to find the main settings we needed. However, less obvious configuration settings may be difficult to find because of the nature of this busy console.

It took us some time to become familiar with the console. At first, users may be disoriented even when trying common Windows conventions because a few of these conventions may not work with this console. For example, numbers cannot just be typed into the correct port dialog box in QPro�s setup screen. Instead, the Dialer, which is the button link to the channel setup, must be used. Also, having the ability to right-click on every featured object would be beneficial to users. Most of the right-clicking functionality we noticed was for the layout design of the console or for copying and pasting, not for guiding users to configuration/monitoring settings.

We were able to perform PAMS and other measurements through the Cisco 186 adaptor from the QPro to the H.323Sim. Running these tests required a few configurations to the H.323Sim but this was done with little difficulty. Our test calls were running within minutes of finishing the configurations. The PAMS scores for Cisco�s gateway averaged close to 3.5 for incoming listening quality and slightly less than 4 for incoming listening effort. These scores signify some degradation but are acceptable (5 would be a perfect score). More importantly for our purposes though is that the test consistently worked and accurate testing information could be attained.

Unfortunately, we had problems negotiating between SIPSim and the SIP proxy server we were using, so we could not measure actual real-time SIP calls. However, we did run tests within SIPSim itself to analyze the inner workings of the application. In addition, we saw that MediaPro reported the errors of calls between SIPSim and QPro, at least showing us that the SIPSim server and MediaPro�s server were communicating with each other. While this did not give us the full scope of how MediaPro functioned, we did receive a demo so that we could receive a better indication of how that application monitored the VoIP signaling data and how it worked in tandem with the rest of the suite. Figure 3 shows MediaPro in action.

Next Version
Much of what we would have detailed as room for improvement issues will actually be available in the next version of the Performer. These new capabilities include a stressing mode for the QPro, which allows simulation of a multitude of calls to stress a gateway or PSTN element while conducting audio quality measurements, and a new capturing utility, which we had the opportunity to look at in beta format and will be available via the Performer Console. There will be remote management capabilities for the Performer, such as running an IP configuration, restart, shutdown, and time zone adjustment remotely. New MediaPro features include online analysis for packet-loss in addition to the online analysis of jitter that is already available, automatic instead of manual voice quality measurements, advanced filters and sorting mechanisms, and a Find feature. A sniffer converter will allow for the conversion of Network Associates� Sniffer trace files so that MediaPro can analyze them. In addition to PAMS, the PESQ quality measurement algorithm will also be available, and users will be able to select the algorithm they would like to use. Lastly, to complete the integration of the entire Performer suite, H.323Sim will be incorporated into the Performer Console.

While there is other equipment and applications that may cover certain aspects of VoIP testing better than RADCOM�s Performer, few if any of them can match its complete functionality. The integration of all of the applications gives any vendor or service provider the opportunity to analyze, monitor, and measure most aspects of VoIP equipment, and it does so using a solid, albeit not perfect, console. We recommend customers looking for a complete VoIP testing solution to inquire more about the Performer. It may just be what they need.

[ Return To The March 2002 Table Of Contents ]

Today @ TMC
Upcoming Events
ITEXPO West 2012
October 2- 5, 2012
The Austin Convention Center
Austin, Texas
The World's Premier Managed Services and Cloud Computing Event
Click for Dates and Locations
Mobility Tech Conference & Expo
October 3- 5, 2012
The Austin Convention Center
Austin, Texas
Cloud Communications Summit
October 3- 5, 2012
The Austin Convention Center
Austin, Texas