May 2001

Agilent Technologies
395 Page Mill Road
Palo Alto, CA 94306
Tel: 800-698-0061
Fax: 303-754-4802

Price: starts at $24,000. The J1981A with J1984A upgrade (VQT Telegra R with analog and T1 interface) is $51,000.

RATINGS (0-5)
Installation: 5
Documentation: 5
Features: 4.5
GUI: 4.5
Overall: A

INSTALLATION
Since the Telegra VQT always comes with the hardware and software preinstalled, there is not much to say about the installation process. Essentially, it was unlatch the keyboard, plug in the power cord, and boot up the computer. Of course, we also needed to connect either the T1 or analog connections, depending on which we were using (both cards were preinstalled as was an Ethernet card in case we wanted to connect to our LAN). The system came with the software on a backup CD just in case there was something wrong with the preinstalled software. We didn't have such a problem, but went through the software installation to see how the process worked. It turned out to be a typical software installation that only took a few minutes.

Configuring the port settings for either the analog or T1/PRI interfaces (an E1 connection uses yet another interface) was not difficult. To set the analog ports, we only needed to accept and apply the default settings. If we wanted, we could have selected different ports and configurations, such as the signaling type and off hook to dial delay, but the default settings suited us just fine. Setting the T1/PRI ports took a little knowledge of what line coding, framing, and signaling type was used for our network. Since we use the Gordon Kapes T1 simulator for the network, we kept its settings and made sure that the Telegra VQT matched these settings except for making the VQT set to the user side.

DOCUMENTATION
Both the printed materials and the help files effectively tell users about the Telegra VQT. The Getting Started Guide gives the user the general idea about all of the tests the product has to offer. It also displays plenty of screenshots and a decent index. Accompanying this guide is a book that focuses on IP telephony in general and a CD-ROM that includes more information about Internet telephony, sample capture files of the major VoIP protocols (H.323, SIP, MGCP), and live examples of VoIP traffic. The book and the CD-ROM are certainly a welcome addition to the Telegra VQT documentation.

While the help files are not quite context sensitive, they may be at least as impressive as if they were. When you first access the Telegra VQT interface, a Welcome screen pops up so that you can review the help files before exploring the VQT interface. When using the interface, contents, index, and search help files are also available. Perhaps more advantageous is the information available on the bottom right corner of every screen, which details the important information particular to the features of that screen. It is an embedded help mode in which you can choose to display the basic or advanced help information and click on the appropriate information you need. You may also choose to hide these embedded files from appearing on the screen. This method proves to be a very effective means of learning about each individual voice quality test.

FEATURES
The primary features of the Telegra VQT involve testing VoIP packet systems for voice quality information and the equipment (gateways, switches, etc.) that makes up a VoIP network. These tests include Perceptual Speech Quality Measure (PSQM), Perceptual Analysis Measurement System (PAMS), Echo-PACE, and Echo-DTalk scores. Other tests includes DTMF tone detection and voice activity (sometimes called silence suppressors) detection. Audio testing tools for noise and tone generation can be applied to .wav files that can be played back and recorded. Automated test sequences can also be implemented for further analysis of VoIP calls.

OPERATIONAL TESTING
First, we examined and tested the T1/PRI interface. Once we hooked up the VQT with the Gordon Kapes simulator and we synchronized those settings, we set four digit numbers in the Call Control screen and placed a simulated call. From here, we ran a test that measured the delay of the call as well as a test for PSQM scores, which measures the clarity of the call. Because of the nature of the T1 test we set up, the scores were extremely low, which means that there was almost no delay and that the call was extremely clear. This result is to be expected testing a regular T1 call from a simulated lab environment.

Next, we wanted to test VoIP calls to obtain more genuine examples of how the Telegra VQT would typically perform. We used two PUREDATA RT 201V Internet Telephony Gateways (ITG), one containing a four-port FXO module and the other containing a four-port FXS module. We set up the gateways and a dialing plan so that we could call into the FXO module of the ITG through an analog Teltone simulator. From the analog simulator, we connected a line to the first FXO port on the VQT. From the VQT GUI, we entered "5551000,,,,,201", which dialed into the first ITG VoIP gateway and then paused the few seconds necessary to connect. Afterwards, the DTMF digits "201" were transmitted, causing the ITG's dialing plan to route the call to the second ITG gateway across the IP network. The second ITG gateway received the IP call and then rang extension 201, which we actually connected back to the VQT to complete the loop. With this loopback in place, the VQT was capable of transmitting sounds as well as receiving those sounds, and thus, it was able to calculate the difference in time (latency) or sound quality degradation (PSQM or PAMS).

Through this method, we could test for delay, PSQM, PAMS, and other scores that are associated with VoIP. Keep in mind that the lower the PSQM score, the better. Below 3 for VoIP calls is considered very good. On the other hand, the higher the PAMS scores, the better (with the highest score being a 5). Obtaining dial tone through the Teltone simulator, the PUREDATA ITG's scores tested with an average delay of ~121ms, average PSQM of ~2.77, and an average PAMS scores of ~4 for listening quality and ~4.3 for listening effort. Over all, the scores for PUREDATA's gateways indicate quality VoIP calls. For more information about the PUREDATA RT 201V Internet Telephony Gateway, please read their review.

There was latency introduced by the Teltone simulator, which we needed to measure. However, we tested this simulator by hooking it up to the VQT system and discovered that the latency introduced was virtually negligible at a miniscule 0.125ms. As a reference, the Teltone simulator had a PSQM score of ~1.05, which indicated excellent quality, and therefore, most likely did not affect the VoIP scores. Remember, however, that the PSQM scores are non-linear and are attained using complex algorithms. Trying to "subtract" the Teltone's PSQM scores from a gateway's actual scores would not give an accurate figure. Nevertheless, the PSQM figures attained using a Teltone PSTN simulator is extremely close to the "true" PSQM value of PUREDATA's ITG when testing on lines with only small amounts of voice degradation or bandwidth issues.

We compared our test results for the analog Teltone simulator with the Gordon Kapes T1 simulator. As expected, the T1 scores showed better quality than the analog scores because of its digital connection. For instance, the average analog PSQM score was 1.05 while the average T1 PSQM score was a superb score of 0.21. Since the PAMS of the Teltone simulator do slightly affect the overall PAMS score of PUREDATA's VoIP gateway, we also tested for these scores. The simulator's excellent PAMS scores averaged a Listening Quality Score of 4.45 and Listening Effort Score of 4.64.

In addition, we also used the VQT to test a "real world" PSTN call using regular analog lines. While the delay was almost non-existent, scoring the same as the Teltone simulator, the PSQM scores were actually a little higher and thus slightly worse than the VoIP scores that used the Teltone simulator. However, keep in mind that when we used regular analog lines to call the local gateway in each direction and calling from gateway to gateway via IP, these VoIP calls averaged more than a point higher than they do testing through an internal simulator because of traffic and voice degradation on the PSTN lines. The PAMS scores of typical PSTN calls were slightly better than the PUREDATA ITG scores through the Teltone simulator and averaged nearly a half-point better than the VoIP tests using PSTN lines (Table 1).

Furthermore, we tested VoIP calls at ConvergeNET using the VQT. Since ConvergeNET is an interoperability showcase of live VoIP calls from one company's booth to another one, these tests were especially important. The PSQM results averaged below 3, proving that quality VoIP calls can be made even on a busy network. The tests were successful even when using two different vendors' H.323 gateways. This certainly showed promise for both VoIP testing and for VoIP interoperability.

ROOM FOR IMPROVEMENT
For testing Internet telephony gateways, the Telegra VQT is the top of the line. Our suggestions involve testing other Internet telephony equipment more easily. We could not test Microsoft NetMeeting or the IP phones we had here in the lab because the VGT requires that the calls must be initiated and terminated on one of VQT's telephony boards (T1/E1/PRI or analog) as opposed to initiating or terminating a VoIP call from an IP connection. We think that if Agilent provided a virtual H.323 gateway on the unit, it could both initiate and terminate VoIP calls via an Ethernet connection, thereby allowing virtually any Internet telephony equipment to be tested. This would benefit the VQT and the entire Internet telephony industry. From talking with Agilent representatives, there may very well be a product to support the VQT in this way in the near future -- maybe even by the time this review is in print.

While the graphical interface is admirable, we would like to see the analog, T1, and E1 interfaces integrated. Currently, for example, we need to disconnect a T1 test and the interface in order to open the analog interface and conduct another test. It might be advantageous to be able to run both tests concurrently or at least be able to schedule a specific time to run the tests one after another. Integrating the GUIs would help towards that end. On a more minor note, right-click functionality would also aid usability.

CONCLUSION
The Telegra VQT is a sophisticated and intuitive piece of testing equipment. Specifically catering to the Internet telephony market, it delivers accurate scores for VoIP elements of gateways, routers, and switches. Any company in this industry as well as the labs testing the equipment would find the VQT as an extremely valuable resource. With the help of this resource, we will see VoIP gateways and the like consistently improving so that the quality of the calls will soon be of equal quality with close to the same rate of delay to that of the PSTN

[ Return To The May 2001 Table Of Contents ]