Fault Resilient Computing for Telecom

We’ve all been told that, in the new IP and World Wide Web world, everything is software. To a great extent that’s true. Pure software approaches allow for remarkable flexibility and a speedy time-to-market. Ultimately however, software does not run in midair, but on some kind of hardware. The concept of communications applications running on reliable applications servers (or dedicated “communications servers” — a term revived by Motorola) is becoming a recognizable paradigm.

But first, a bit of history... Once upon a time (the 1980s) a profusion of PBX makers appeared following the dismantlement of AT&T. Flush with cash and notoriety, they thought that their devices — proprietary card cages accompanied by so-called OAIs (Open Application Interfaces, a misnomer if there ever was one) — would continue to expand in capability. Indeed, they actually believed that increasingly computerized PBXs would become the new “mainframes” that would control the office, computing data and moving it around between departments, as well as provide interesting new forms of call control. A few phone systems appeared that could actually allow data to travel over the phone system wiring (present day modems are an example of how to send data over voice). IBM/Rohm placed computerlike phones on the desktop, but a combination of complex functions, expense, and limited utility kept such systems from taking over the enterprise.

One concept that did survive from that era was applying computer intelligence to phone calls. This time, however, the computer “smarts” would be running on PCs. As early as 1983, Nick Zwick, one of the founders of Dialogic Corporation, began designing the first, open multiline voice board for PCs. The following year, Hank Magnuski developed the first computer-based fax board and founded GammaLink, Inc., which ultimately became part of Dialogic.

In 1985, a DEC (Digital Equipment Corporation) employee named Carl Strathmeyer developed the first modern computer-PBX link. Strathmeyer, who would later become a marketing director for Dialogic, devised the acronym CIT (Computer Integrated Telephony). IBM became interested in the technology, but didn’t want to use a term devised by its chief competitor, DEC. Jim Burton, then at IBM, obliged by switching some letters of the acronym around to get CTI. Ironically, Burton has joked on many occasions about how he always liked Strathmeyer’s term better than his own.

Even so, the public latched onto “CTI” and it was the principal term used to describe the technology until expo entrepreneur Marc Ostrofsky used the term “Voice Processing.” This term was quickly usurped by when Harry Newton and Gerry Friesen began promoting the fateful phrase “Computer Telephony” in 1993 with a magazine and expo of the same name.

In 1989 Dialogic introduced the first 12 channel DSP-based voice processing board, the first T-1 interface board for voice processing, and the first digital TDM (define - news - alert) bus for resource sharing (PEB). Other companies, such as Natural MicroSystems, Brooktrout (now Cantata), Amtelco, PIKA, NewVoice (now part of AudioCodes) followed suit, joining the fray with their own “CT boards.”

As a result, all kinds of amazing new business opportunities appeared to tempt developers and integrators: IVR systems, fax servers, debit card systems, international callback systems, inexpensive PC-based phone systems (called “CT Servers” or “unPBXs,” a term coined by Ed Margulies) with voicemail and automatic call distribution, and call centers using PC-based predictive dialers and PC-based databases furnishing “screen pops” of customer data to the PC screens of customer service representatives. At the high end, telcos were becoming concerned as their connect rates became more competitive, and they realized that the only way they could seize new customers and hold on to them was to offer “enhanced services” such as one-number “follow me” dialing, which requires a computer platform.

As the first computer telephony developers scrambled to develop their solutions, many of them amusingly plugged their CT boards into inexpensive mail order-type PCs. Since revenuegenerating services must ideally run 24- hours a day, it soon became starkly evident that your typical cut-rate PC doesn’t have the “stamina” to handle these kinds of applications for very long. Plug too many voice and fax cards into a PC and the typical small power supply bombs out. Thrash a single hard drive 24 hours a day and you’ll soon be losing all of your data.

Carriers, major service providers, and multinational enterprises settled on buying computer equipment having the same kind of reliability and high availability enjoyed by their digital switches. They relied on the standard, extensive set of performance, quality, environmental, and safety requirements developed by Telecordia called NEBS (Network Equipment — Building System). Such NEBS-certified, fault tolerant computer systems were capable of “five nines” (99.999%) system uptime, but there were few computers with such characteristics and those that existed were expensive. Motorola sold its “FT” or “Fault Tolerant” proprietary computer to telecom equipment companies such as Nortel, and there were similarly high-priced systems by Stratus and Tandem, which also used special proprietary software and were underpowered by today’s standards, thus reducing their cost/benefit ratios.

Around 1993, everybody grasped that the heavy duty “industrial computers” then appearing were rugged enough to be reconfigured for telecom use. These 19-inch wide rackmounts had passive backplanes (with 20 or so slots for plug-in boards), redundant storage (RAID), load-sharing power supplies and many forced air fans or “blowers,” but unlike true fault tolerant systems, they had only one CPU on a single board computer (SBC). A single CPU meant that their uptime was diminished to “four nines,” but it also made it less expensive and thus suitable for affordable IVR, fax, voicemail, and other computer telephony systems. In 1995, while at Computer Telephony magazine, Yours Truly coined the term “fault resilient computer” to describe this heretofore unclassified device, which ended up powering the 1990s computer telephony boom.

One of the first companies to build such “telecom rackmounts” or “communications servers” was Dialogic itself. Dialogic’s NEBS-certified Fault Resilient Telco Platform (DTP/FR) was a 19- inch rackmount rugged enough for both computer telephony developers and the central office environment. It came with a 20-slot ISA passive backplane or optional 18 slot (14 ISA / 4 PCI) platform — with built-in alarm capabilities that met the requirements of Bellcore’s LATA Switching Systems Generic Requirements (LSSGR) for fault detection and alarm notification.

Telecom engineers still tended to sneer at such rackmounts, since they expected components to be “hot swappable” (swapping out failed boards or other components while the system is running). Clearly a new kind of computer form factor (hardware) had to be developed.

In early 1994, Ziatech (later part of Intel, now part of Performance Technologies) conceived of a new type of PCI-bus rackmount that could bring relatively inexpensive, open architecture, highly efficient, reliable, and easily upgradeable equipment to both circuit and packet switched networks. Ziatech and six other companies developed the original specification in 1994-1995. It was electrically similar to the PCI bus in desktop computers, but was ruggedized for telco-related use and supported hot swappabiliy. Ziatech’s original name for this was “RuggedPCI” and it was limited to the 3U-high form factor. At a September 1994 meeting of PICMG (PCI Industrial Computer Manufacturers’ Group), Dennis Aldridge, a marketing fellow from Texas Microsystems (now part of RadiSys) complained about the name “RuggedPCI,” saying that his passive backplane products were already rugged. Joe Pavlat, then running a company called Pro-Log (later part of Motorola) instantly came up with the term “CompactPCI” at that meeting. It was only intended to be a temporary name, but it stuck. Engineers like to abbreviate things, and Wayne Fischer, then of Force Computers (now part of Motorola), began using “cPCI” in correspondence. It too became popular, although the correct term is, of course, “CompactPCI.”

CompactPCI was expected to be the great hope of telecom, bringing desktop software running on a mass-produced, standardized, ruggedized PC to telcos for a reasonable capital investment. Problems appeared however. It took years to pin down the cPCI spec; then the telecom bubble burst. Also, people realized that the PCI bus itself was still a single point of failure and had bandwidth limitations. While PCI still has a lot of life left (since PCs can be clustered to boost system capacity and reliability) the need for greater computing power and bandwidth has led to the development of new, more powerful form factors, AdvancedTCA (Advanced Telecom Computing Architecture) and its baby brother, MicroTCA. This has also led to taking the PCI bus that ran between the boards on the backplane and replacing it with new high bandwidth (and highly reliable) “switch fabrics” such as 10 Gbps Ethernet, PCI Express, Advanced Switching, RapidIO, and Infiniband.