June 2001

BY JEFF LAWRENCE

History
For years, Peripheral Component Interconnect (PCI) and its variants, H.110, and other proprietary bus architectures were the workhorse technologies used to build computing and communications equipment for the network infrastructure. PCI was originally designed as a low-latency parallel, shared-bus architecture to interconnect microprocessor, I/O, and memory chips. Over time, it evolved into not only a chip-to-chip, but also a board-to-board interconnect technology.

The open architecture and economies of scale realized from the dramatic volumes in the client and server space created compelling reasons to use PCI in the communications space. PCI was a good, but not perfect, solution for communications. It didn't support isochronous traffic very well, was bandwidth-limited, and was unable to meet the high-availability requirements of communications equipment, including hot swap and hot plug-in. PCI evolved to address some of these problems, and the H.110 bus arose independently to address the support of isochronous traffic.

Parallel to the development of PCI and H.110, Ethernet and Fiber Channel became the standards for the local area network (LAN) and storage area network (SAN) to interconnect clients, servers, network appliances, network attached storage, and routers. These technologies are packet/message-based, and what they give up in latency they make up for with reach and scalability. Though the chip-to-chip and board-to-board technologies have been evolving independently from the LAN and SAN technologies, all four technologies are currently on a collision course with each other.

Requirements
Future computing and communications architectural models will be about transporting, cracking, and stuffing packets at wire and fiber speed. Ten Gbps will become the standard unit of bandwidth in the NGN as 10 Gigabit Ethernet, 10 Gigabit (OC-192) fiber, 10 Gigabit Fiber Channel, and InfiniBand proliferate. New processing models will be developed to support these requirements. It is likely these models will consist of interconnected processing elements, ranging from application processors to ASICs, which can provide a continuum of optimized processing for the different phases of an application or service. The old way of doing things will simply not work for the future. The techniques for switching and routing in network elements will migrate into chips, and, in principle, the network will become a backplane interconnecting the processing, storage, and transport elements on a scale ranging from inches to thousands of miles.

Traditional interconnect technologies were designed for low bandwidth (that is, less than 1 Gbps) and bursty traffic. In the future, interconnect technologies will need to support streaming high-bandwidth traffic that may be distributed in a point-to-point or multipoint manner in either the electrical or optical domain. Data and communications centers are starting to use very high-density and high-performance equipment that consumes power on the order of hundreds of Watts per square foot. Servers, network appliances, and other types of equipment will need to be disaggregated (separated into compute, communications, I/O, and storage elements) to meet the high power density and high-availability needs of the future.

Interconnect technologies will need to scale from low-cost clients to large distributed systems that support memory or packet-based I/O models to meet stringent cost, performance, scalability, density, and availability requirements for computing and communications equipment. Any new interconnect technology will also need to transparently support the large installed base of PCI driver and application software to preserve the huge investment made by service providers, network operators, and equipment manufacturers.

We are about to enter a period of technological disruption as long-established bus technologies are succeeded by new serial, low pin count, low voltage differential signaling, and high bandwidth solutions designed to meet computing and communications needs for the packet-based infrastructure. Leading candidates for this disruption include third generation I/O (3GIO), HyperTransport, and Rapid I/O (RIO) for chip-to-chip and board-to-board interconnect, and 1/10 Gbps Ethernet and InfiniBand for board-to-board interconnect and SAN.

Future
There is no clear leader yet, and the stakes are high. Intel initially announced 3GIO in the spring of 2001. HyperTransport (initially known as Lightening Data Transport) is being driven by AMD, while Motorola is driving Rapid I/O (RIO). These interconnect technologies have many similarities and some differences. None of these technologies is deployed in production systems, and success will be determined not just by the perceived technological superiority of one solution over another, but also by the formation of a critical mass of technology (intellectual property cores, software, components incorporating the technology, bridges, etc.), companies, and people able to bring these to market.

Fiber Channel is well suited for the SAN environment. Ethernet will expand out from the LAN and become an important transport technology for the access infrastructure, Metropolitan Area Network (MAN) infrastructure, Storage Area Network (SAN), and even the board-to-board interconnect across the backplane. Ethernet and its variations are well-understood technologies, have pervasive industry support, and have very attractive economics, all of which bodes well for their increased usage in the transport infrastructure. InfiniBand, by comparison, is new and designed to overcome many of the technical limitations of Ethernet, and is particularly well suited to support the needs of the disaggregated or virtual server. The technical advantages of InfiniBand will also enable it to be used in many other SAN and board-to-board interconnect applications.

PCI, 3GIO, HyperTransport, and RIO are suited for processing-centric models and products such as clients or devices that consist of a small number of tightly coupled nodes communicating with each other via memory. PCI, 3GIO, HyperTransport, RIO, Ethernet, and InfiniBand are suited for communications-centric models and products consisting of a large number of loosely coupled nodes that communicate with each other by packets. These models will demand high performance, high scalability, high availability, and high density. They will not be particularly cost-sensitive and will likely be the service platforms, servers, network appliances, content switches, and softswitches of the NGN.

Conclusion
The computing and communications industries are about to enter a transition period, during which a number of technology and business issues will play themselves out as the interconnect technologies for the NGN are settled upon. There is the potential for fragmentation, but the pressures for open, highly scalable, and widely embraced interconnect technologies are great. Computing and communications will become inextricably linked as we go through this transition, and the technologies settled upon will define the future direction and potential of products for at least the next five to ten years. Application and product developers will need to educate themselves about the possibilities and limitations of the various interconnect technologies so they can influence and then use the new technologies to develop the services and applications that will meet the promise of the NGN.

Jeff Lawrence is part of Intel Corporation's Intel Communications Group. Prior to Intel, Jeff co-founded Trillium Digital Systems, Inc., a provider of communications software solutions to equipment manufacturers, and served as its president and CEO until Intel acquired the company in August 2000.

[ Return To The June 2001 Table Of Contents ]