Europeans Build 10-GigE Network

UTRECHT -- CERN, SURFnet and the University of Amsterdam announced that they have succeeded in building and testing a Trans European 10 Gbps Ethernet (10 GE) network. Crossing four countries and spanning 1700 km, the network uses the new 10 GE WAN PHY transmission technology capable of transmitting the equivalent of 1.5 complete data CDs every second. The CERN Amsterdam connection was used by researchers from the ATLAS experiment and the University of Amsterdam to transfer a single TCP data stream between a pair of high-end server PCs at 5.4 Gbps, a rate limited by the PCs hardware and software. Using two PCs at each end it was shown possible to utilize the full data bandwidth (9.2 Gbps) of the CERN to Amsterdam connection. The results are an important milestone in exploring the high-speed distribution of data from CERN's new Large Hadron Collider, scheduled to come into operation in 2007.

The network consisted of a SURFnet OC-192 lambda between Amsterdam and Geneva delivered by Global Crossing. The setup was first tested for 90 hours using IXIA network testers. This equipment proved the stability and raw performance of the network by transporting 365 TBytes of raw data without any bit error or packet loss. In subsequent tests Force10 switch/routers were connected directly to the DWDM equipment using two 10 Gbps WAN PHY interfaces. Another setup included Cisco ONS 15454 Multiservice Optical Platforms in the path. For test purposes several HP Itanium and Xeon systems were connected on both sides equipped with Intel 10 Gbps Ethernet network interface cards.

The University of Amsterdam and high energy physics research institute NIKHEF collaborate with SURFnet to build advanced data transport infrastructure using lightpaths. This infrastructure will aid the distribution of the experiment data from the LHC experiments at CERN to researchers in remote institutes via the data replication centres of which NIKHEF will be a node.

The 10 GE WAN PHY technology permits Ethernet frames to be carried across existing SONET/SDH infrastructures. It effectively extends the span of an Ethernet network across countries and continents. Long haul Ethernets of the type demonstrated between CERN and Amsterdam can offer significant cost advantages as they use cheaper and fewer components than traditional alternatives, in particular when the 10 GE WAN PHY is directly connected to DWDM equipment, as has been shown in some of these tests. The Ethernet everywhere approach nicely augments the emerging optical demonstrators at NetherLight, StarLight and CANARIE, and the existing router based general Internet infrastructure.

The current successful demonstration of the 10 GE WAN PHY in the field has been built on earlier interoperability testing carried out by CANARIE, Carleton University and CERN in the laboratory this summer.

Bob Dobinson from CERN said: "Over the last decades we have seen Ethernet become the predominant local area network standard. Now we see the use of Ethernet at the Trans European level. "Ethernet everywhere" is a not inappropriate slogan for the future world of networking. This could be a significant development for the ATLAS experiment."

Kees Neggers, Managing Director of SURFnet, applauds the results. "SURFnet is pioneering with 'hybrid' networking since early 2002 via the NL GigaPort project. This work by CANARIE, CERN and University of Amsterdam is an excellent demonstration of the potential of lambda services for science applications and yet another step in the development of GLIF, the emerging Global Lambda Integrated Facility."

Cees de Laat, associate professor at the University of Amsterdam comments: "This is a very important step in proving the feasibility of building a services oriented network where adequate equipment is used to deliver the best price-performance for the requirements of the GRID users. Using router-based infrastructures only at these speeds would turn out to be too expensive."

Bill St. Arnaud, CANARIE commented: "The present tests are a step towards extending the Internet end-to-end principle to circuit-based networks. Soon high-end GRID applications will have sufficient traffic volume to require their own lightpaths, complementing the standard routed network. Ethernet over lightpaths is a technology that may allow significant cost savings while providing an increased level of flexibility for such scenario."

The tests were made possible by support from the following manufacturers, who have generously contributed their equipment and know how: Force10, IXIA, Hewlett Packard, Cisco, Intel, Global Crossing. We acknowledge the support of: EU projects ESTA (IST-2001-33182) and DataTAG (IST-2001-32459), NL GigaPort project, CERN OpenLAB, NIKHEF and SARA.

SURFnet

CERN

University of Amsterdam