U of Tokyo Breaks Records for Ethernet

PITTSBURGH -- Researchers from the University of Tokyo and the Japanese WIDE Project, together with engineers in Japan, Canada, the United States, the Netherlands and Switzerland completed the world’s longest 10 Gigabit per second circuit ever recorded for the transmission of internet data. The high bandwidth link connected geographically dispersed servers from the University of Tokyo’s Data Reservoir project stretching from the Supercomputing 2004 research exhibition in Pittsburgh, Pennsylvania to the CERN research center in Geneva, Switzerland through Tokyo. The length of this fiber optic path is approximately 31,248 km, spanning 17 time zones. The link was used to perform high-speed TCP data transfers that will lead to unprecedented breakthroughs in collaborative physics and engineering experiments between dozens of research institutions worldwide without distance limitations.

In the experiment, 7.21 Gigabits per second TCP payload bandwidth was sustained on a single stream with standard 1500 Byte Ethernet packets between two servers connected by 31,248km network. This international cooperative project pushes the boundaries of global research and education and lays the foundation for a new array of international research opportunities.

Using 10 Gigabit networking technology that combines OC-192 Packet over SONET technology and 10 Gigabit Wide Area Networking technology, a local area network connecting computers at the University of Tokyo’s SC2004 Exhibition booth in Pittsburgh, PA was extended to include computers at CERN in Switzerland connecting through SCinet, Abilene, JGN2 and APAN, Tokyo. The network from APAN to T-LEX was provided by the WIDE project. From T-LEX, the circuit was passed to Seattle using wavelength donated by Tyco Telecommunications through the IEEAF, and cross-connected through facilities provided by Pacific Northwest Gigapop in Seattle. From Seattle the circuit was then carried across a dedicated lambda on the CA*net 4 network to the Chicago Starlight. At Starlight, the interconnect to SURFnet’s Chicago-Amsterdam lambda was made, taking the connection to Netherlight in Amsterdam. Finally, between Netherlight and CERN, SURFnet’s Amsterdam-Geneva lambda was used.

The data transfer is achieved between a pair of data-sharing Opteron systems from the Data Reservoir project, one server placed at the SC2004 exhibition booth of the University of Tokyo and another at CERN, each equipped with a Chelsio T110 10 Gigabit Ethernet adapter supporting TCP/IP offload. Transfer rate of 7.21 Gbps was sustained for over 15 minutes using a single TCP stream and standard 1500-byte Ethernet frames over the 31,248km link. The combined bandwidth times distance value is a new world record at 225,298 terabit meters per second and is 80% greater than the previous Internet2 Land Speed Record of 124,935 terabit meters per second. At this transfer rate and distance, a full-length DVD can be transferred anywhere on the earth in less than five seconds.

The Data Reservoir system also achieved a 1.6 Gbps disk-to-disk transfer with a single quad-Opteron server with a Chelsio T110 TCP offload engine at each end of the connection. This performance figure shows that 200 Mbytes per second single box disk servers are readily available for a wide range of researchers.