Light Reading

The 400Gbit/s Demonstrator

Eduard Beier
8/21/2013
100%
0%

A collaboration between research institutions and industrial partners demonstrated at the ISC'13 that 400Gbit/s bandwidth granularity is not only feasible, but already useful today.

For demonstration purposes a number of innovative technologies such as 400Gbit/s DWDM Super Channel, high-speed flash memory and a distributed parallel file system are used in combination.

The project is set up to have real data and realistic applications instead of yet another "hero experiment" with test generators and analyzers. Initial performance tests were performed to check the operational condition of all components working together in the demonstrator. Synthetic data and load was put on the connectivity and IT components and showed overall good operational conditions.

Then two applications were activated on the demonstrator:

  • Climate Research with centralized computing accessing to distributed data

  • Turbine Development with multi-stage processing dynamically both shifting big-data between Munich and Dresden (620km standard fiber) over a single 400Gbit/s wavelength Super Channel.

    From the research community are involved: the German Centre for Aerospace (DLR); the Leibniz Supercomputing Centre (LRZ); the Max Planck Institute for Meteorology (MPI-M) from the Max Planck Society and the computing center of the Max Planck Society in Garching (RZG); the computer center of the Technical University of Freiberg (RZ TUBAF); and the Center for Information Services and High Performance Computing Dresden University of Technology (ZIH).

    Industrial partners are: Alcatel-Lucent (NYSE: ALU), Barracuda Networks Inc. , Bull SA , Cluster Vision, EMC2, IBM Corp. (NYSE: IBM), Mellanox Technologies Ltd. (Nasdaq: MLNX), Deutsche Telekom AG (NYSE: DT), and T-Systems International GmbH .

    The fundamental network structure of the Demonstrator is shown by the following picture:

    Network Structure

    The compute clusters on both sides are clients of the distributed parallel file system which consists of 12 servers with three flash memory boards in each server. Both clusters are able to read and write at 400 Gbit/s on that file system.

    Standard Sandy Bridge servers with at least five PCIe3.0x8 slots and 128GB DRAM are used. The design goal was to create about 18 GBytes of sustained duplex data rate per node (Ethernet: 5GBytes, Memory: 6 GBytes, Fabric: 7 GBytes). The Fabric data rate has not yet been tested, the other rates have been confirmed by tests.

    For performance margin reasons the setup is moderately overbooked between memory and network. Below are the total theoretical performance numbers.

    Theoretical Throughput

    The predecessor project in 2010 (100Gbit/s Testbed Dresden Freiberg) required arrays of about 800 spinning disks on each side to form a 100Gbit/s data path. Because that setup was, for scaling reasons, not possible for 400 Gbit/s, the 2 GBytes I/O throughput flash memory board is one of the enablers of this 400Gbit/s project.

    For the ISC configuration (in the picture below), the following systems were added: a 10GbE Cluster in Dresden; a 2x100GbE link to Freiberg (not connected during ISC); in Garching the SuperMUC at LRZ (one of the fastest supercomputers in Europe) and a 480 GBytes cluster (> 2500 cores) at RZG; and a commercial Cloud Service from T-Systems (not connected during ISC).

    The Big Picture

    The project links to SDN and NFV -- in particular as it is very active in NSI (Network-to-Service-Interface) definition. (For further information, see the link below to additional project information).

    Because the "distributed high speed GPFS" approach is of some universal nature (e.g. for HPC datacenter backup and HPC workload distribution reasons) the setup will be tested for commercial applicability during the post ISC phase. The possibility to use network functions like encryption, firewalling and data compression is definitely a must in a commercial case.

    Network appliances for 40 Gbit/s and 100 Gbit/s are neither available nor would they be affordable in many cases. Therefore we are going to test virtualized network functions on standard server hardware (see picture below); that additional "module," which is based on the same server hardware as the other servers, gets in between each server and the router on each side.

    NFV Module

    Please feel free to ask questions about any technical aspect of the project. The project team, which consists of some of the best specialists on their individual field, will be happy to answer. For additional project information: http://tu-dresden.de/zih/400gigabit.

    The Partners

    — Eduard Beier, process systems engineer, T-Systems International

    (6)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    RolfSperber
    50%
    50%
    RolfSperber,
    User Rank: Light Beer
    9/26/2013 | 10:11:00 AM
    Re: SDN and NFV
    Ray, we are at a very early stage, but abstracting from both hardware (and in consequence IOS) layer and utilizing a common framework (see NSI WG in OGF) and at the same time allow for docking of virtalized network functions will work for multi vendor, multi carrier and multi domain. Still, its a long way to go!
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    9/26/2013 | 7:51:13 AM
    Re: SDN and NFV
    Rolf

    You say it is not restricted to a single domain, but is it applicable in networks that traverse multiple infrastructures run and managed by multiple network operators?
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:43:49 AM
    Re: Is this another route in for NFV?
    Requirements in industry will not be so different from those in R&D. Looking at the plans in the context of Horizon 2020 Puplic Private Partnership is a target of European efforts. Taking into account the cost of infrastructure and the prevailing attitude to pay as little as possible for utilization of infrastructure, more sophisticated multiplexing methods, and this include NFV, are inevitable.

    For network operators this scheme means significantly reduced time to market.
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:36:55 AM
    SDN and NFV
    In this project we will be going a step further. Our plan is to create an environment that allows for creating a virtual network based on the requirements of either applications or  carrier provided network functionality. We will not be restricted to a single domain and connectivity, our target is a network created from building blocks out of a repository. These building blocks can be connectivity with certain quality parameters or virtualized network functions like e.g. firwall functionality, compression, encryption, accelleration.

     

     
    Eddie_
    50%
    50%
    Eddie_,
    User Rank: Blogger
    8/21/2013 | 3:00:47 PM
    Re: Is this another route in for NFV?
    NFV simply scales better than HW based approaches (if NFV tests in September show good results).

    A possible 2013 roadmap for the project:
    • fully SDN controlled network
    • 200GBit/s datapath
    • NFV in the data path 

    a possible 2014 roadmap for the project:
    • scale up to a 1TBit/s data path

    how else could you do that?

     

     

     

     

     
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    8/21/2013 | 1:01:52 PM
    Is this another route in for NFV?
    Interesting that functions virtualization takes the place of appliances that are either too expensive to deploy or have not yet been created... is NFV going to help the R&D sector more than production network operatins in the early years?
    Educational Resources
    sponsor supplied content
    Educational Resources Archive
    More Blogs from Column
    The industry is at an inflection point, and one of the looming disruptors is 5G.
    Network architects aiming to upgrade their networks to support agile, open, virtualized services in the 21st century need to consider new criteria when choosing between technology suppliers.
    Cable companies still dominate the scene at INTX, the event formerly known as The Cable Show.
    What is the best way to support both layer 2 and layer 3 services at the edge of the network that incorporates virtual network functions?
    How fast is too fast in terms of communication? Unless our lives and actions start being measured in seconds, do we really need a 43 Tbps connection to fulfill our everyday tasks?
    Flash Poll
    From The Founder
    Last week I dropped in on "Hotlanta," Georgia to moderate Light Reading's inaugural DroneComm conference – a unique colloquium investigating the potential for drone communications to disrupt the world's telecom ecosystem. As you will see, it was a day of exploration and epiphany...
    LRTV Documentaries
    Verizon's Emmons: SDN Key to Cost-Effective Scaling

    5|22|15   |   03:53   |   (0) comments


    For Verizon and other network operators to ramp up available bandwidth cost effectively, they need to move to SDN and agree on how to do that.
    LRTV Documentaries
    Lack of Universal SDN a Challenge

    5|21|15   |   04:51   |   (3) comments


    Heavy Reading Analyst Sterling Perrin talks about how uncertainty about SDN standards and approaches may be slowing deployment.
    LRTV Custom TV
    Steve Vogelsang Interview: Carrier SDN

    5|20|15   |   05:02   |   (0) comments


    Sterling Perrin speaks to Steve Vogelsang, Alcatel-Lucent CTO for IP Routing & Transport business, about the new Carrier SDN-enabling Network Services Platform and the operator challenges it solves.
    LRTV Custom TV
    Carrier SDN: On-Demand Networks for an On-Demand World

    5|20|15   |   20:52   |   (0) comments


    Steve Vogelsang, Alcatel-Lucent CTO for IP Routing & Transport business, talks about requirements and benefits of Carrier SDN during the keynote address at the Light Reading Carrier SDN event May 2015.
    LRTV Documentaries
    The Security Challenge of SDN

    5|19|15   |   02:52   |   (0) comments


    CenturyLink VP James Feger discusses concerns that virtualization could create new vulnerabilities unless network operators build in safeguards.
    LRTV Custom TV
    NFV Elasticity – Highly Available VNF Scale-Out Architectures for the Mobile Edge

    5|18|15   |   5:50   |   (0) comments


    Peter Marek and Paul Stevens from Advantech Networks and Communications Group talk about their NFV Elasticity initiative and the company's latest platforms for deploying virtual network functions at the edge of the network. Packetarium XL and the new Versatile Server Module: 'designed to reach parts of the network that other servers cannot reach.'
    LRTV Huawei Video Resource Center
    Bay Area Spark Meetup 2015

    5|14|15   |   3:54   |   (0) comments


    Developed in 2009, Apache Spark is a powerful open source processing engine built around speed, ease of use and sophisticated analytics. This spring, Huawei hosted a meetup for Spark developers and data scientists in Santa Clara, California. Light Reading spoke with organizers and attendees about Huawei's code contributions and long-term commitment to Spark.
    LRTV Custom TV
    The Transport SDN Buzz

    5|12|15   |   06:01   |   (1) comment


    Sterling Perrin, senior analyst at Heavy Reading, speaks with Peter Ashwood-Smith of Huawei and Guru Parulkar of ON.Lab about the evolution of transport SDN and the integration of technologies.
    LRTV Custom TV
    Next-Generation CCAP: Cisco cBR-8 Evolved CCAP

    5|5|15   |   04:49   |   (0) comments


    John Chapman, Cisco's CTO of Cable Access Business Unit and Cisco Fellow, explained the innovation design of Cisco's cBR-8, the industry's first Evolved CCAP, including DOCSIS 3.1 design from ground-up, distributed CCAP with Remote PHY and path to virtualization. Cisco's cBR-8 Evolved CCAP is the platform that will last through the transitions.
    LRTV Custom TV
    Meeting the Demands of Bandwidth & Service Group Growth

    5|1|15   |   5:35   |   (0) comments


    Jorge Salinger, Comcast's Vice President of Access Architecture, explains how DOCSIS 3.1 and multi-service CCAP can meet the demands of the bandwidth and service group growth.
    LRTV Custom TV
    DOCSIS 3.1: Transforming Cable From Hardware-Defined Network to Software-Defined Network

    4|29|15   |   03:48   |   (0) comments


    John Chapman, Cisco's CTO of Cable Access Business Unit and Cisco Fellow, explains how DOCSIS 3.1 can transform cable HFC network to a more agile software-defined network.
    LRTV Huawei Video Resource Center
    Predicting Traffic Patterns for Quality Mobile Broadband

    4|29|15   |   6:45   |   (0) comments


    Accessing information ubiquitously creates complexity and creates heavy traffic onto the network, especially at large-scale events like sporting events or festivals. In this video, Huawei's Mohammad Hussain speaks to experts about how to predict traffic and improve user experience during periods of heavy traffic.
    Upcoming Live Events
    June 8, 2015, Chicago, IL
    June 9, 2015, Chicago, IL
    June 9-10, 2015, Chicago, IL
    June 10, 2015, Chicago, IL
    September 29-30, 2015, The Westin Grand Müchen, Munich, Germany
    October 6, 2015, The Westin Peachtree Plaza, Atlanta, GA
    October 6, 2015, Westin Peachtree Plaza, Atlanta, GA
    All Upcoming Live Events
    Infographics
    Network functions virtualization (NFV) is not the easiest of topics to take on board, so here's a Light Reading infographic, developed following conversations with the folks at HP, that helps make sense of where NFV is taking the industry.
    Hot Topics
    Verizon Saves 60% Swapping Copper for Fiber
    Sarah Thomas, Editorial Operations Director, 5/19/2015
    Choosing a Technology Supplier? Consider Changing Your Selection Criteria
    Steve Saunders, CEO and founder, Light Reading, 5/18/2015
    Chattanooga Charts Killer Gigabit Apps
    Mari Silbey, Senior Editor, Cable/Video, 5/20/2015
    10 Alternate Uses for Tablets
    Eryn Leavens, Copy Desk Editor, 5/22/2015
    Smarter 'Dumb' TVs Will Drive OTT Adoption
    Mari Silbey, Senior Editor, Cable/Video, 5/18/2015
    Like Us on Facebook
    Twitter Feed
    Webinar Archive
    BETWEEN THE CEOs - Executive Interviews
    With 200 customers in 60 countries, Stockholm-based Net Insight has carved out a solid leadership position in one of the hottest vertical markets going in comms right now: helping service providers and broadcasters deliver video and other multimedia traffic over IP networks. How has Net Insight managed to achieve this success in the face of immense competition from the industry giants?
    My ongoing interview tour of the leading minds of the telecom industry recently took me to Richardson, Texas, where I met with Rod Naphan, CTO and SVP, Solutions, ...
    I recently popped down to Texas to chat with CEO Eric L. Pratt about his company, Taqua.
    Cats with Phones