Planning For The Worst To Ensure The Best
Disaster Recovery in Telecommunications

BY BRIAN FABIANO

For businesses, a highly reliable telecommunications network is a high stakes issue. A single day without telephone, data and Internet service can translate into millions in lost profits and damaged customer relationships. For some customers, the consequences of a network interruption are more dire -- a matter of life and death, even. Hospitals and county 9-1-1 emergency systems, for example, can’t risk a single minute of network downtime without endangering the health and welfare of the communities that they serve.

It’s an inescapable fact of life that disasters happen. And it’s an ironic fact that the very communications links that can mitigate the sometimes-devastating effects of those disasters are often severely compromised by the event itself. While many disasters can’t be prevented -- just try and stop an earthquake, hurricane or flood -- their impact on telecommunications networks can certainly be lessened, if not eliminated entirely, when the proper precautions and business continuity plans are implemented.

For customers, whether small, large or enterprise, this can be as simple as knowing what to ask their service provider and understanding how to interpret the company’s answers.

As one of the few carriers in the New York metropolitan area to achieve nearly 100-percent reliability during the Aug. 14, 2003, blackout, Lightpath, the Cablevision-owned local exchange carrier serving New York City, Long Island, Westchester County, northern New Jersey and southern Connecticut, has some hard proof of the effectiveness of its disaster-recovery and business continuity planning. Given the success it has experienced to date, Lightpath’s approach to “bulletproofing” network communications can serve as a what-to-look-for guide that businesses can refer to in planning their own disaster-recovery measures.

When evaluating their carriers’ ability to weather a disaster smoothly, customers should closely examine the infrastructure design and construction, how network equipment is selected and tested and the company’s history of providing high levels of service. If the carrier can meet these requirements, then chances are good that your business and its network can withstand any interruption.

THE NETWORK: BULLETPROOF BY DESIGN
The first, and most important, line of defense against network disaster is in the way a carrier’s underlying infrastructure has been designed and built. Customers need to ask their carriers a number of probing questions to definitively determine whether the network can deliver the reliability and survivability most businesses and enterprises require. Ideal design configuration spans a broad array of criteria that encompasses everything from fiber counts to conduit pathways and construction processes.

One of the first infrastructure issues a customer should examine is redundancy. While most carriers will offer a client the option of signing up for separate services or circuits, if the redundant paths they’re purchasing aren’t truly diverse, then they’re not truly redundant. If the provider in question shares infrastructure with other providers, as most do, they may be riding fiber on the same sheath as another company or in the same interduct or conduit. If that bundle were to suffer some damage, then it doesn’t matter how many separate strands of glass or paths a customer has purchased because they’ll all be affected.

A provider with solid disaster-recovery provisioning will own the right of way to the conduit and will be the only operator in that trench. True diversity means geographical -- not just electrical -- diversity. The most robust disaster-resistant approach will ensure that a customer’s redundant paths are geographically dispersed and far enough apart so that a single event cannot possibly affect both paths.

Truly diverse paths also require the equipment the paths terminate on to be diverse as well, and, ideally, each node should have redundant cards in it to prevent a single point of failure. Those redundant nodes should then be placed in separate central offices or head-ends. Sometimes, even the optical multiplexers within those head-ends can be redundant as well. If a carrier provides dual feeds into its client’s office and offers truly diverse routing to different head-ends, it’s next to impossible to lose service.

The conduit construction methods are also important to review. Many carriers have not buried fiber deep enough to avoid construction-related outages. Cable should be placed at least three feet down and the interduct should be within the conduit and the conduit covered with concrete and steel. It’s a first line of defense and an indispensable one. Customers should grill potential carriers on the protective measures that go into network construction.

Another important reliability consideration in network design is whether the carrier adheres to a “first mile to the last mile” approach. Some legacy network providers use copper wire for the crucial last or first mile stretch between the local exchange network and a customer premise, but running a fiber backbone directly into a customer’s business eliminates this weakest link that would be overly vulnerable during a disaster-related event.

EQUIPMENT: DOES THE CARRIER BUY THE BEST AND TEST, TEST, TEST?
To minimize the possibility of equipment failure during times of network stress, a carrier should be asked if it deals only with best-in-class equipment vendors. But equipment security doesn’t stop with a best-of-breed purchase. That equipment should be subjected to the carrier’s own rigorous testing and must conform to its own high in-house standards. This is an often overlooked, but important, consideration in disaster-proofing the network.  Simply, what may be fine for one provider may not be the right choice for another. Every piece of equipment should be tested in an environment that imitates a long-haul network to certify that the product does what it says it will and performs to the highest standards. Highest standards mean that every component of the network should be held to five-nines (99.999 percent) -- not three-nines or four-nines -- reliability.

SERVICE, SERVICE, SERVICE
If the infrastructure is redundant, truly diverse and incorporates tested, failsafe equipment, that’s only two-thirds of the battle. The best approach can be ruined by poor customer service.

To ensure that a carrier will respond promptly and effectively to service outages and other problems, customers should look for a number of network-managed services. First and foremost, does the carrier in question have multiple points of monitoring? With multiple points of monitoring, the carrier will never be blind to its network’s service levels, and presumably can identify, address and oftentimes, even prevent potentially problematic issues directly from the main network operations center at all times.

Another helpful indicator is provisioning and mean-time-to-repair metrics. If a carrier consistently comes in lower than industry benchmarks, and below its own targets on the time it takes to deploy a new service or repair an existing one, then it’s probably safe to assume the company has robust service levels. Less tangible, but equally important is whether a carrier embraces a customer-first philosophy.  The measures of a carrier’s service level can be found simply by observing the technicians, sales force and other company representatives. Well-informed, competent and responsive employees are better able to -- and more likely to want to -- provide excellent, caring service than those who seem ill trained or unhappy. It’s a simple, but worthwhile, point.

Disasters happen. They often happen without warning. But that doesn’t mean your communications providers should be caught unaware next time catastrophe strikes. If the carrier meets the design, equipment and service mandates outlined here, chances are good that the network, and your business, will come through the worst unscathed.

Brian Fabiano is senior vice president, Network Services, for Lightpath. The company provides a dependable, fail-safe and innovative foundation for business communications needs in New York, New Jersey and Connecticut.  Lightpath owns, installs and operates its own advanced fiber-optic network facility, comprising over 10,000 route miles of fiber-optic cable that connects nearly 1000 commercial buildings.

Respond to this article in our forums!