10-Gigabit Ethernet

The IEEE802.3ae 10-Gigabit Ethernet has been developed from earlier Ethernet standards.

The original Ethernet standards worked over a shared medium, i.e., a single cable connecting all the nodes together. Starting with 10Base-T, the network changed from a single cable to a star topology with a repeater at the center. Although there were now two wires between each node and the repeater, the network was still a shared medium and the total network bandwidth was only 10 Mbit/s. By replacing the repeater with a switch, 10Base-T can support full duplex traffic – 10 Mbit/s in each direction between each node and the central switch. The total network bandwidth is now the sum of the links to and from the switch, although the bandwidth may be limited by the switch’s internal architecture. With 100-Mbit/s Ethernet and Gigabit Ethernet, the bandwidth on each port of a switch is set following auto-negotiation between the node and switch. Some ports support 10-Mbit/s, 100-Mbit/s or 1-Gbit/s Ethernet.

A key part of the Ethernet protocol has been the CSMA/CD collision detection mechanism used to arbitrate for the shared media. Ten-Gigabit Ethernet is the first Ethernet technology to become entirely full duplex and therefore does not support CSMA/CD.

To accelerate time-to-market for Gigabit Ethernet the Gigabit Fibre Channel PMD (Physical Media-Dependent) layer was used for the optical interfaces. The 10-Gig Ethernet standard has now leapfrogged over Fibre Channel development, and therefore several new optical PMDs have been developed. Now the 10-Gig Fibre Channel standard is expected to use a 10-Gigabit Ethernet PMD.

Gigabit Ethernet uses the 8B/10B coding to include additional symbols at the PMD layer. Unfortunately, this increases the signal frequency to 12.5 GHz. To achieve the 10-Gbit/s bandwidth and limit the signal frequencies as close to 10 GHz as possible, a new, more efficient 64B/66B coding was developed. Ten-Gigabit was originally defined as an optical-only network, but there is now work in the Institute of Electrical and Electronics Engineers Inc. (IEEE) to support copper media (see Startups Move 10-Gig to Copper).

A significant goal for 10-Gigabit Ethernet has been to stretch the range from the 5 kilometers in Gigabit Ethernet to 10 and 40 km. There are now a number of companies developing proprietary optics to take Ethernet to 80 km, 100 km, and beyond.

There has been some deployment of Gigabit Ethernet in the WAN. A significant hurdle, however, has been the different data rates supported by Gigabit Ethernet and 622-Mbit/s or 2.5-Gbit/s Sonet/SDH. Unlike all other Ethernet standards, 10-Gig Ethernet is close to a Sonet/SDH data rate (OC192/STM64). To improve compatibility further, the IEEE has defined the WAN PHY, which will interface directly with 10-Gbit/s Sonet/SDH networks. The WAN PHY includes a WAN interface sublayer (WIS), which is a cut-down Sonet framer.

Table 1 shows the main differences between 10-Gigabit Ethernet and Gigabit Ethernet.

Table 1: Gigabit vs 10-Gigabit Ethernet

Gigabit Ethernet	10-Gigabit Ethernet
CSMA/CD + full duplex	Full duplex only
Leveraged Fibre Channel PMDs	New optical PMDs
Reused 8B/10B coding	New coding schemes 64B/66B
Optical/copper media	Optical media only
	(copper in development)
Support LAN to 5 km	Support LAN to 40 km
Carrier extension	Throttle MAC speed for WAN
	-�� Use Sonet/SDH as Layer 1 transport

Ten-Gigabit Ethernet has the same layers as other Ethernet technologies: Media Access Control (MAC), Physical (PHY), and Physical Media-Dependent (PMD). Ten-Gig Ethernet also supports all the existing Ethernet technologies, such as group trunking, and is therefore simple to introduce into an existing Ethernet-based network.

Interfaces between the layers have been standardized. The 10-Gig Ethernet MAC is connected to the PHY through either the XGMII interface or the low pin-count alternative, XAUI. The XAUI interface has 4x 3.125-Gbit/s channels with 8B/10B encoding. XAUI can also be used to connect the PHY to the PMD.

The IEEE has defined a total of seven optical port types for 10-Gig Ethernet; these are shown in Table 2, together with two proposed standards, for copper cable, shown in italics.

Table 2: IEEE 802.3ae Port Types

Device	Application	Range	Optics	Cable
10GBase-LX4	LAN	300m MMF/	1310nm	Multimode or
		10km SMF	WWDM	singlemode
10GBase-SR	LAN	300 m	850nm	Multimode
10GBase-LR	LAN	10 km	1310nm	Singlemode
10GBase-ER	LAN	40 km	1550nm	Singlemode
10GBase-SW	WAN	300 m	850nm	Multimode
10GBase-LW	WAN	10 km	1310nm	Singlemode
10GBase-EW	WAN	40 km	1550nm	Singlemode
10GBase-CX4	LAN	15m	-	4 x Twinax
10GBase-T	LAN	25-100m	-	Twisted pair

The 10GBase-LX4 interface has four low-cost lasers and supports both multimode (MMF) and singlemode (SMF) fiber. There are three versions of both the LAN and WAN interfaces supporting multimode fiber for short distances and singlemode fiber for longer distances. The 10-Gigabit Ethernet distances are defined as 300 meters for short reach (SR), 10 km for long reach (LR), and 40 km for extended reach (ER).

“Most enterprises have pulled mixed cable plant,” says Force10’s Quiros. “So they have both multimode and singlemode fiber in their backbone. That has given them a lot of flexibility in the way they can deploy 10-Gigabit Ethernet.”

The majority of 10-Gig Ethernet deployment so far has been in the campus backbone and between campuses. The 10Gbase-LR and 10GBase-ER are the most suitable interfaces for these applications. Pre-standard 10-Gig systems from Cisco featured a proprietary interface called 10GBase-EX, which supports a 50km range.

Within the IEEE there is now work to develop two copper interfaces. The first, 10GBase-CX4, uses existing InfiniBand IB4X connectors and four Twinax cables. The second, 10GBase-T, is a totally new development to support 10-Gigabit Ethernet over 25 to 100 meters of standard twisted-pair cable.

10GBase-CX4 borrows much from Infiniband and is designed for short reach, data center applications. By using existing connectors and cables, the standards group expects to achieve full ratification by the end of 2003.