When you take a look at all the exciting developments in the communications space — a daily visit to TMCnet will keep you abreast of all the happenings in the industry — it’s sometimes hard to imagine that all the innovative products and solutions that are being rolled out to business and residential customers work reliably. Yet, they do — though sometimes upgrades come with a few wrinkles that need to be ironed out.

 

To avoid the hassles of patching holes in security measures or fixing other slight defects in programming, perhaps the most important aspect to developing new solutions or upgrades is the testing process. Providers and vendors alike have to test for protocol and services interworking, performance, capacity, network strain, and more, because they need to ensure any glitches or shortcomings are identified before the new product or solution is rolled out to the user community. Once that happens, it is both costly and embarrassing to fix problems.

 

Fortunately, there is a wide array of test solution vendors that provide all the necessary equipment and software to test effectively. “Zippy Grigonis’ feature from the June issue of INTERNET TELEPHONY takes a look at the communications test space.

 

Less fortunate, however, is that none of the solutions perform all the necessary testing functions equally well. 

 

“Some are good for call generation, but not so good at feature testing; others are very good at feature testing, but aren’t the best for voice quality testing, and so on,” explained Terry Caterisano, President and CEO of CallMation. “So, you often end up with a variety of test equipment from a number of different vendors in test labs over time”

 

While that may be something of an inconvenience, it’s something providers and equipment manufacturers have come to terms with. However, what presents a bigger challenge is a comprehensive tool for automating testing across these various vendors’ products, something that would allow testers to automate many of the repeated procedures that are required with new product releases and software upgrades.

 

That’s the challenge Caterisano says CallMation allows testers to overcome with its WerkMATE solution.

 

Most test equipment vendors, he acknowledges, include some level of automation with their solutions, but not enough. More importantly, though, the automation they do enable isn’t compatible with other vendors’ products. WerkMATE, on the other hand, sits on top of all the various test solutions in a lab, providing a vendor agnostic solution that runs not only the test equipment, but also handles devices under test and other ancillary equipment, as well as data collection and storage of scripts and results in a central repository.

 

Importantly, because many companies have already developed some of their own scripting, CallMation’s solution allows them to leverage that existing work, enabling those scripts and programs to be run through and stored in its automation framework as well.

 

The information repository is a key component, because it not only stores scripts and test data that can be easily accessed via a Web interface, eliminating the need for manual sharing of information across distributed organizations.

 

“We’ve allowed users to store all the information about the tests inside our test framework,” said Caterisano. “The scripts are located there, as well as the procedures, all the other information and documentation about the testing.”

 

Now, as for the ability to manage testing across different vendors’ solutions, Caseritano says there really aren’t any standards yet. Most vendors provide some level of automation and remote control over their devices, but only for the very basic functions, like loading a script, starting it, stopping it, and capturing results. The problem is that there is no consistency between vendors as to how they enable those capabilities. 

 

CallMation has developed a means to normalize those functions, and all others, effectively eliminating the need for users to understand the different operating details for their different equipment. It does this through a series of adapters developed to connect the WerkMATE software to test equipment, allowing users to use generic command sequences to automate testing. The specific handling of proprietary interfaces is done behind the scenes, along with error handling and exceptions.

 

CallMation has also developed a generic adapter that allows users to script their own device interfaces, so they don’t necessarily have to use the pre-built adapters. The adapters, though, have been developed with the intent of relieving much of the burden associated with running test equipment from different vendors. Test equipment vendors generally make the specifications for their command interfaces available, largely because CallMation isn’t viewed as a competitive threat.

 

“They aren’t really in this space,” said Caterisano. “It’s up to third parties, like CallMation, to develop these innovative solutions.”

 

In fact, he notes that, if a user is able to enter commands into a terminal to test devices or software, that test solution can be automated with WerkMATE. However, he is quick to note that this not a complete replacement for manual testing. 

 

“There will always be a need for manual testing,” he says. “But, if I can offload 50 percent of the work I do on a daily basis onto a framework that runs tests overnight and on weekends, I’m ahead of the game.”

 

To read more about CallMation and what Caterisano has to say about the testing world, read his blog, Tomorrow’s Testing Today, on TMCnet.