QoS in Enterprise Networks

Quality has always been an obsession with IP Communications equipment vendors and network operators. It's a veritable industry unto itself. With the PSTN, you either have a single, crystal clear, exclusive connection to your called party, or you don't have a connection at all (as explained by the network recording, "All circuits are busy, please try again later.") With IP, you can cram ever more calls into the same pipe, but the tradeoff is that the quality of each conversation suffers — unless you're using a technique for prioritizing important realtime voice and video traffic, such as Cisco's MPLS — and even then quality will collapse if you continue to increase the call volume way beyond the pipe's bandwidth capacity. Packet latency (packets arriving late), packet loss, and jitter (variation in packet arrival times) are the three principal variables that can be used to measure how well a particular call flow or even an entire IP Communications system is doing. But this isn't the whole story of what makes for a quality call.

Because of quality concerns, in the early days of VoIP, not much voice traffic was going over the (then) wild and wooly Internet. Instead, most "one wire wonders" of the 1990s took PSTN calls, converted them to IP, and sent them over the (hopefully) more stable and controlled environment of the corporate IP data LAN. The so-called "network PBXs" of the time didn't follow any particular standard architecture, but there was generally an IP PBX router with an Ethernet backplane that allowed calls (in the form of IP packets) to enter and leave the system from the PSTN (and in later years, the Internet) and travel right onto the LAN, hanging off of which at intervals were (and still are) Ethernet-compatible IP phones.

Such early systems demonstrated how even a modicum of IP technology could simplify the enterprise wiring-plant and all the things associated with it such as maintenance, MACs [Moves, Adds, Changes] and other management issues, thus placing many early adopters in a position to be among the first to reap the benefits of IP telephony.

However, even restricting IP traffic to the enterprise network doesn't necessarily guarantee crystal-clear voice and/or video quality. Today's corporate networks are more complicated than ever before, with VPN connections to branch offices and teleworkers, IP peering with enterprise partners and customers, and WiFi access points dotting the LAN landscape, all of which can cause packet traffic congestion and affect call quality on the LAN.

Benjamin Ellis, Vice President of Global Marketing at Psytechnics (news - alert) (http://www.psytechnics.com) says, "Psytechnics’ history was really with testing IP communications Quality of Experience [QoE] with service providers and carriers and test measurement sites, but in the last year we've done a great deal of work with enterprises, owing to the fact that lots of businesses have finished experimenting with VoIP and are now starting to switch over in a wholesale manner to VoIP, and they want to ensure QoS across the network, but they're running into problems, so they call us. Our testing work is now balanced between service providers and enterprises."

"These days there are multiple dimensions to QoS and where it gets implemented," says Ellis. "For an enterprise, there are many questions. What's the QoS arrangement of the carrier? What's the IP QoS in the network? What's happening in the Ethernet LAN? And so forth. Much of this was fairly theoretical until about a year ago. People were still playing with the technology, but they weren't really depending on it. Now, however, there are major VoIP systems sold by vendors such as Cisco, Avaya, and unified communications is on the rise, which adds video into the communications mix. People are also in a videoconferencing equipment upgrade cycle, under the banner of 'telepresence'. So, suddenly, the network managers find that their theory has been tested, as it were, and in a lot of cases it has been found wanting, but for lots of different reasons."

"Many companies use traditional network tools and when most people talk about QoS, they're referring to what we would call network-based or link-based QoS," says Ellis. "If you've got a service provider supplying a T-3 [45 Mbps] between two different offices, you might have an SLA [Service Level Agreement] that says that there's less than five percent packet loss. But what the SLA doesn't specify is which five percent of the packets get lost. That sounds funny, but oftentimes people will mark voice packets as high priority traffic and everything else as low priority, so one would expect the voice to get a high QoS, since there's only a five percent packet loss. But a five percent packet loss can mean lots of different things. It could mean that you lose 10 packets in a row and then the next 180 packets arrive in perfect order; that would cause a glitch or drop-out that the user will hear in the phone call and will find quite disruptive. On the other hand, you could lose one in 20 packets, which is also a five percent loss but wouldn't be noticed by the user."

Ellis continues: "Moreover, if there are multiple calls traveling across that link, particularly now with switches that work with flows, a five percent loss might translate into a loss of five percent of the flows, which means that five percent of all phone calls — entire phone calls — can be lost."

"So, traditional ways of even defining QoS for IP networks don't really map into things such as telephony or videoconferencing, because what you're interested in there is the quality of an individual stream of packets comprising a conversation," says Ellis. "When you're looking at bulk file transfers, resending packets is not such a big deal. But in the case of telephony and videoconferencing, that directly touches users, so when something goes wrong, they see it and complain. One of the CIOs we work with describes users as his most expensive netnetwork management tool when it comes to VoIP, because they are the ones who alert him to a problem with a barrage of five or more angry phone calls from, say, the New York office, complaining about calls 'breaking up', and yet the network management system is indicating that everything is fine. Like other organizations, they monitor VoIP and video, and because they are streams from user-to-user you need management tools that monitor and manage streams on network links."

"From what we've seen," says Ellis, "there are two camps of customers. One camp managed to get some QoS in place and they're moving from the theory of QoS to the practical implementation. They've discovered that, if you're doing session-based protocols for voice and video, you need to monitor on a slave or 'streams' basis. The second camp involves people revising their idea of QoS, which is the traditional definition of QoS, a definition that's very network centric, and it involves the IP layer. That's fine for applications where you're only really concerned about what happened on your network, because the IP networks do a great job of moving data from one place to another and generally they don't 'mess up' what's inside of the packets. However, IP voice networks are fundamentally different because your voice network is of restricted value unless it connects to another network."

"So, you have to connect to either the PSTN, the traditional telephone network," says Ellis, "or you have to do what's increasingly happening now, something that's certainly common in Europe and is starting a bit in the U.S. What I'm referring to is a company having a connection that goes from the VoIP network out to the PSTN, to a telephony provider, and they might connect to a different provider for international calls to get a better tariff, and they might connect with another provider to get a better rate for cellular phones, and directly connect to business partners in an effort to hold onto them."

"This is all very exciting," says Ellis, "because it results in much more costeffective telephony, which was one of the reasons for adopting VoIP in the first place. However, the big challenge for the corporate IT department is that communications is a whole new ballgame because now their users get affected by what's happening in other networks. The PSTN's problems were ironed out decades ago. But now you're introducing a whole set of different places and ways from which voice quality problems can emerge. In once case we know of in the U.K., a company was suffering from call problems and it turned out that there was one particular location where they paired their voice network in such a way that a small percentage of their calls were getting disrupted. For a business that depends on transactions with customers done over the phone, even a small percentage can impact on the viability of the company. If five percent of users hang up because they can't hear what the customer service representatives say, then that costs the company real money over time."

"The challenge is that VoIP networks are turning out to be a lot more complicated than a traditional telephony network, because you've got all of these various points where other network technologies are connecting. Aside from LAN QoS and WAN QoS, you've also got a whole set of application quality issues that don't really come under the traditional banner of QoS. Even stream or session-based QoS monitoring schemes still sometimes run into problems, because to effectively manage VoIP and telepresence you need to manage beyond your network. This relates to a real situation concering one of our government customers. Their really good IP QoS tools were all showing 'green lights'. Everything appeared to be perfect. But users were still complaining about call quality. As it turned out, they had several hidden issues. One was that they had some poor quality handsets sitting in one part of the network. Another issue was the software running on their media gateway was incorrectly configured. On another gateway, the echo cancellers had been turned off. So you had these problems, and yet the IP QoS tools determined that everything was okay, and from an IP point of view it was. But, of course, that's not the whole picture. You still have to manage voice."

Ellis elaborates: "Managing voice itself is a new thing for enterprises because telephony has been so stable for so long, so when you talk about QoS for voice, most people tend to look at you in a confused manner or else they start talking about IP QoS factors, such as packet loss and jitter parameters. These certainly affect telephony quality but even if they're perfect, you can still encounter quality problems inside of the whole system, such as echo, delay, incorrect speech levels and distortion, background noise, and so forth. All of these things can ruin a phone call, and yet a traditional QoS toolset is oriented toward IP phenomena that govern data more than voice quality."

So it appears that even in the secure, controlled world of the corporate LAN, companies such as Psytechnics can still do a booming business helping enterprises find the multitude of otherwise obscure problems that affect voice and video quality.

And how are your IP phone calls sounding today?

Richard Grigonis is Executive Editor of TMC's IP Communications Group.