×

SUBSCRIBE TO TMCnet
TMCnet - World's Largest Communications and Technology Community

CHANNEL BY TOPICS


QUICK LINKS




 
NGN Magazine Magazine logo
Jan/Feb 2009 | Volume 1/Number 1
On the Testing Edge

After Interoperability What’s Next?

By Andy Huckridge

We recently saw the conclusion of the Multiservice Forum’s GMI2008 event and an ETSI IMS Test event. In both of the test events, the by-and-large outcome is that Interoperability has now been tested-in. What test methodologies are next being considered?

Sometimes, everyone is right.

NGN networks rely on more and complicated software components as the building blocks of complex networks, or service offerings. From the above events it can be seen that we largely have the interoperability issue licked, at least for those vendors who want to test against each other that is. But now the testing paradigm is evolving & changing; today we are exposing more software blocks directly to the end-user, testing the actual software in more unforeseen ways. When testing software, the purpose is to validate correct behavior – there are only two possible outcomes to a test, either the software functions as it is supposed to function, or it does not. A third possible outcome is that the test itself is incorrect. But sometimes, things are not this straightforward. There is Only One Way




Staff involved in test design often mentions to me that for a test to be meaningful you always need to know the correct response to each Stimulus sent to the Device Under Test (DUT). For tests utilizing a UI, the inputs can consist of mouse movements and characters received from the keyboard. For protocol tests, the inputs are often defined as a sequence of requests and responses. For each input, there is a set of valid responses.

A protocol is an agreed format for communications, much the same way that a conformance test audits the functionality of an implementation against a specific version of a protocol specification. For each functionality, there is a set of message sequences that exchange the predefined set of data to reach that functionality. A state diagram describes the valid state transitions and the different messages that are sent from one party to the other to reach the next state in the communications process. A valid message sequence traverses this state diagram according to the protocol design.

My Way or the Highway

Protocol conformance can often result in an argument. When products of two different development groups do not communicate between each other, the developers start pointing fingers at either one of the implementations. The protocol specifications often leave space for interpretation. Protocols are still just agreements, and the purpose is interoperability between vendors. But not always. Sometimes the purpose of protocol specifications is just marketing. Some vendors have no interest to be interoperable. They can dictate standards, and if they so desire, write new ones.

Only the Destination Matters

A network that consists of products of one vendor only can appear to be extremely interoperable, and fast. There is little reason to use all the connections between complex sequences of messages if you can reach the same result with fewer messages. You can make shortcuts. Other shortcuts relate to usability. It seems to be perfectly valid to respond to VoIP call initiation with a “Ringing” message, even if in reality there is not yet anything really ringing on the other end. To the consumer, this will create a false impression of effectiveness and operation where there is none.

You Will Decide the Correct Way


All the above arguments have their supporters, for test tool vendors it is also not easy to test, since any valid response is often ok. Only the well trained user of the piece of test equipment will determine that outcome. Sounds suspicious — right? Well it is — but only for conformance test methodology. It is not suspicious if no specification exists. In real life, it is impossible to define all possible requests and responses to software.

In the Negative, Robustness & Security test methodology the purpose is to verify the service / DUT stays up and the associated software does not crash. An intelligently broken message can result in extremely weird responses. Sometimes a request results in an error message. Sometimes the broken message is passed through and accepted as a valid request. And oftentimes the request is just silently discarded. All these being perfectly valid functionality. Good testing!

Andy Huckridge is Vice President, Marketing, Codenomicon. Andy has worked in the Silicon Valley telecommunications industry for more than a decade and has a broad background in defining and marketing products for the semiconductor, VoIP and IMS/NGN space. Reach him at [email protected]

NGN Magazine Table of Contents









Technology Marketing Corporation

2 Trap Falls Road Suite 106, Shelton, CT 06484 USA
Ph: +1-203-852-6800, 800-243-6002

General comments: [email protected].
Comments about this site: [email protected].

STAY CURRENT YOUR WAY

© 2023 Technology Marketing Corporation. All rights reserved | Privacy Policy