As carriers, enterprises and government organizations continue their migration to unified IP-based computing and communication systems, they must also “armor up” their networks against an increasingly hostile environment, inhabited by a growing number of rapidly-evolving threats. Data systems designers and network architects have adopted layered, integrated security standards such as ITU X.8051, which specify an interlocking set of functions that deliver critical capabilities, including:

• Authentication, Authorization, and Accounting (AAA);
• Confidentiality (typically addressed by encryption);
• Data integrity (making sure the data was not modified);
• Availability (usually targeted by denial-of-service attacks); and last, but not least;
• Physical (building access and protection from destruction) security.

Fig.1: In order to protect themselves and the distributed applications they support, next-generation networks’ security strategies must include provisions for cloud and time security to the data security provided by conventional networks.

Traditional data centers and other applications, which rely primarily on dedicated network connections, can meet many of these requirements by encrypting networks’ data, management and control traffic. But as we shall see, encryption doesn’t fully protect the increasingly dispersed and physically accessible network elements of highly distributed systems, such as small-cell wireless networks and smart grids, against attack.

Securing Distributed Systems Raises New Issues

Several recent attacks, which took advantage of open servers on various types of devices attached to a network, have underscored the need for a more rigorous, holistic approach to securing distributed systems. The new issues these next-generation networks bring to the security equation include: 

Cloud Security – Unlike traditional Enterprise IT and networking applications which use well-defined point-to-point connections to reach users in fixed locations, cloud IT services and mobile data networks serve their users as they roam across multiple access points, and even multiple clouds (Fig.2).  Their widely dispersed and physically accessible network elements and machine-to-machine (M2M) communications make it easy to access distributed systems via an ordinary Internet connection. But they also greatly amplify both the potential for, and the consequences of, a network security breach. Similar issues arise in smart grids and other applications based on the “Internet of Things,” which are enabling dramatic improvements in efficiency to nearly every aspect of commercial, civil and private activities.

Fig.2: Unlike conventional IT systems, distributed systems, such as cloud computing and wireless data services, must support secure connections between a wide range of devices across multiple networks, locations and even multiple cloud-based services.

Time Security – The telephony, real-time monitoring/control and streaming multi-media are only some of the network services that require access to network timing information, allowing the network’s widely-dispersed elements to synchronize transactions with each other and keep packets flowing in a predictable manner. Some of the consequences of allowing a network’s timing information to be compromised include:

• Mobile devices have difficulty accessing their network, and when they do, they suffer from dropped calls, poor coverage and low data rates;
• Network time errors in smart energy systems can trigger faults in the power grid; and
• In financial/trading systems, network time glitches can interfere with the smart order routing for large trades which requires simultaneous execution on multiple exchanges.

Data Security – Many government regulations (HIPAA, PCI (News - Alert), Sarbanes-Oxley, Basel-II, etc.) require that medical and financial information are protected with strict, near-military-grade confidentiality, despite the fact that many of their transactions are made across open networks, on publicly-accessible equipment.

In these applications, all data must be encrypted when it leaves the physical security of a building. Although encryption can be done on an application-by-application basis using SSL, TLS or SSH, network-wide encryption is gaining popularity since it eliminates any potential compliance concerns by securing all traffic on the connection.  

Network-wide security can either be applied by the user endpoints, or offered as a service by the network service provider interconnecting various locations, datacenters and other authorized sites. Today’s most commonly-used security protocol is IPsec, which operates at Layer-3 of the Internet Protocol stack. While popular for routed networks, it usually requires dedicated standalone or embedded encryption engines, and cannot encrypt common Layer-2-based control protocols such as ARP, DHCP, Spanning Tree and E-LMI.  

As service providers make Carrier Ethernet Layer-2 VPN services more available, encryption at Layer-2 is becoming an option that is easier to use, lower overhead and easily scales with the number of ports on the network. Layer-2 services also shield the end customer from the network service provider’s routing decisions, so all they see at each remote access node is an extension of the enterprise’s Layer-2 network.

MACsec Protocol Enables L2 Security

To support efficient encryption and security in Layer-2 applications, the IEEE (News - Alert) added the MACsec (802.1AE) and KeySec security and key management/distribution protocols to the IEEE 802.1X pantheon. They provide a robust security solution at the link layer, protecting physically-vulnerable access links such as those external to carrier buildings, as well as exposed inter-box links in mobile backhaul.

Early MACsec silicon implementations only work on a “link-by-link” basis, requiring encryption and decryption at every hop (Fig.3). This can introduce a variable delay of almost 100ns at each link, which can compromise the accuracy of the network time stamps used by the IEEE 1588 v2 Precision Timing Protocol, causing it to exceed the time error budget for most TD-LTE (News - Alert) and LTE-Advanced standards (typically 10-20ns per hop). As a result, MACsec has been deployed in datacenters and other systems whose networks span a limited number of hops, but it has only seen limited adoption by 4G cellular infrastructures and other wide-area service provider networks.

Fig.3: The MACsec (802.1AE) security protocol provides secure L2 connections, but the variable encrypt/decrypt delays introduced at each link can easily exceed the timing budget of 4G wireless data networks and other advanced applications.

New Solutions Enable L2 Encryption and Network Timing Accuracy

Thankfully, distributed systems will soon be able to enjoy both accurate network timing and L2 security as a new generation of MACsec-enabled PHY devices emerges. For example, Vitesse’s Intellisec technology is designed to accurately predict MACSec’s encrypt-decrypt delays and correct the network’s PTP time stamps on a packet-by-packet basis3. Intellisec has already been incorporated into Vitesse Ethernet PHYs supporting full line-rate encryption for both 1GE and 10GE speeds over multiple media types.

With the advent of timing-aware Layer-2 security solutions from Vitesse and other vendors, multi-operator and cloud based networks will soon be able to support highly-secure MEF (News - Alert) CE 2.0 EVCs and MPLS Service Provider connections, independent of the network’s awareness of security protocols. In doing so, these devices will enable PHY-up security in 4G wireless networks and cloud-based services.