& cplSiteName &

The 400Gbit/s Demonstrator

Eduard Beier
8/21/2013
100%
0%

A collaboration between research institutions and industrial partners demonstrated at the ISC'13 that 400Gbit/s bandwidth granularity is not only feasible, but already useful today.

For demonstration purposes a number of innovative technologies such as 400Gbit/s DWDM Super Channel, high-speed flash memory and a distributed parallel file system are used in combination.

The project is set up to have real data and realistic applications instead of yet another "hero experiment" with test generators and analyzers. Initial performance tests were performed to check the operational condition of all components working together in the demonstrator. Synthetic data and load was put on the connectivity and IT components and showed overall good operational conditions.

Then two applications were activated on the demonstrator:

  • Climate Research with centralized computing accessing to distributed data

  • Turbine Development with multi-stage processing dynamically both shifting big-data between Munich and Dresden (620km standard fiber) over a single 400Gbit/s wavelength Super Channel.

    From the research community are involved: the German Centre for Aerospace (DLR); the Leibniz Supercomputing Centre (LRZ); the Max Planck Institute for Meteorology (MPI-M) from the Max Planck Society and the computing center of the Max Planck Society in Garching (RZG); the computer center of the Technical University of Freiberg (RZ TUBAF); and the Center for Information Services and High Performance Computing Dresden University of Technology (ZIH).

    Industrial partners are: Alcatel-Lucent (NYSE: ALU), Barracuda Networks Inc. , Bull SA , Cluster Vision, EMC2, IBM Corp. (NYSE: IBM), Mellanox Technologies Ltd. (Nasdaq: MLNX), Deutsche Telekom AG (NYSE: DT), and T-Systems International GmbH .

    The fundamental network structure of the Demonstrator is shown by the following picture:

    Network Structure

    The compute clusters on both sides are clients of the distributed parallel file system which consists of 12 servers with three flash memory boards in each server. Both clusters are able to read and write at 400 Gbit/s on that file system.

    Standard Sandy Bridge servers with at least five PCIe3.0x8 slots and 128GB DRAM are used. The design goal was to create about 18 GBytes of sustained duplex data rate per node (Ethernet: 5GBytes, Memory: 6 GBytes, Fabric: 7 GBytes). The Fabric data rate has not yet been tested, the other rates have been confirmed by tests.

    For performance margin reasons the setup is moderately overbooked between memory and network. Below are the total theoretical performance numbers.

    Theoretical Throughput

    The predecessor project in 2010 (100Gbit/s Testbed Dresden Freiberg) required arrays of about 800 spinning disks on each side to form a 100Gbit/s data path. Because that setup was, for scaling reasons, not possible for 400 Gbit/s, the 2 GBytes I/O throughput flash memory board is one of the enablers of this 400Gbit/s project.

    For the ISC configuration (in the picture below), the following systems were added: a 10GbE Cluster in Dresden; a 2x100GbE link to Freiberg (not connected during ISC); in Garching the SuperMUC at LRZ (one of the fastest supercomputers in Europe) and a 480 GBytes cluster (> 2500 cores) at RZG; and a commercial Cloud Service from T-Systems (not connected during ISC).

    The Big Picture

    The project links to SDN and NFV -- in particular as it is very active in NSI (Network-to-Service-Interface) definition. (For further information, see the link below to additional project information).

    Because the "distributed high speed GPFS" approach is of some universal nature (e.g. for HPC datacenter backup and HPC workload distribution reasons) the setup will be tested for commercial applicability during the post ISC phase. The possibility to use network functions like encryption, firewalling and data compression is definitely a must in a commercial case.

    Network appliances for 40 Gbit/s and 100 Gbit/s are neither available nor would they be affordable in many cases. Therefore we are going to test virtualized network functions on standard server hardware (see picture below); that additional "module," which is based on the same server hardware as the other servers, gets in between each server and the router on each side.

    NFV Module

    Please feel free to ask questions about any technical aspect of the project. The project team, which consists of some of the best specialists on their individual field, will be happy to answer. For additional project information: http://tu-dresden.de/zih/400gigabit.

    The Partners

    — Eduard Beier, process systems engineer, T-Systems International

    (6)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    RolfSperber
    50%
    50%
    RolfSperber,
    User Rank: Light Beer
    9/26/2013 | 10:11:00 AM
    Re: SDN and NFV
    Ray, we are at a very early stage, but abstracting from both hardware (and in consequence IOS) layer and utilizing a common framework (see NSI WG in OGF) and at the same time allow for docking of virtalized network functions will work for multi vendor, multi carrier and multi domain. Still, its a long way to go!
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    9/26/2013 | 7:51:13 AM
    Re: SDN and NFV
    Rolf

    You say it is not restricted to a single domain, but is it applicable in networks that traverse multiple infrastructures run and managed by multiple network operators?
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:43:49 AM
    Re: Is this another route in for NFV?
    Requirements in industry will not be so different from those in R&D. Looking at the plans in the context of Horizon 2020 Puplic Private Partnership is a target of European efforts. Taking into account the cost of infrastructure and the prevailing attitude to pay as little as possible for utilization of infrastructure, more sophisticated multiplexing methods, and this include NFV, are inevitable.

    For network operators this scheme means significantly reduced time to market.
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:36:55 AM
    SDN and NFV
    In this project we will be going a step further. Our plan is to create an environment that allows for creating a virtual network based on the requirements of either applications or  carrier provided network functionality. We will not be restricted to a single domain and connectivity, our target is a network created from building blocks out of a repository. These building blocks can be connectivity with certain quality parameters or virtualized network functions like e.g. firwall functionality, compression, encryption, accelleration.

     

     
    Eddie_
    50%
    50%
    Eddie_,
    User Rank: Blogger
    8/21/2013 | 3:00:47 PM
    Re: Is this another route in for NFV?
    NFV simply scales better than HW based approaches (if NFV tests in September show good results).

    A possible 2013 roadmap for the project:
    • fully SDN controlled network
    • 200GBit/s datapath
    • NFV in the data path 

    a possible 2014 roadmap for the project:
    • scale up to a 1TBit/s data path

    how else could you do that?

     

     

     

     

     
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    8/21/2013 | 1:01:52 PM
    Is this another route in for NFV?
    Interesting that functions virtualization takes the place of appliances that are either too expensive to deploy or have not yet been created... is NFV going to help the R&D sector more than production network operatins in the early years?
    Educational Resources
    sponsor supplied content
    Educational Resources Archive
    More Blogs from Column
    WiFi is offering a challenge to the network-centric cellular status quo and that's something that mobile network operator CEOs recognize, believes Devicescape CEO Dave Fraser.
    NFV can bring operational headaches as well as operational gains, argues Andy Huckridge.
    5G is about so much more than just very high bandwidth and low latency – and SDN is going to play a key role in enabling 5G full potential, argues ONF Executive Director Dan Pitt.
    To ensure the best possible customer experience, service providers must find ways to make more performance metrics available to users.
    How cable operators can prepare to grasp the mobility opportunity.
    From The Founder
    Download our complete guide to de-risking NFV deployment in 2016, including:
  • An eight-step strategy to deploying NFV safely, based on input from the companies that have already started virtualizing their production networks.
  • Interviews with leading executives at Colt, AT&T, Deutsche Telekom, Cisco, Nokia, ZTE, Ericsson and Heavy Reading.
  • Flash Poll
    Live Streaming Video
    Prepping for the Future: Upskill U Explained
    During this short kick-off video, Doug Webster, Vice President of Service Provider Marketing, Cisco, and Light Reading’s CEO & Founder Steve Saunders give an overview of Upskill U.
    Between the CEOs
    CEO Chat With SCTE's Mark Dzuban

    5|3|16   |   14:02   |   (1) comment


    Mark Dzuban, President and CEO of SCTE, sits down with Steve Saunders in Light Reading's New York office to discuss the society's Energy 2020 campaign, including its mission to benefit the environment, enable economic benefits and the key challenges facing cable operators wanting to reduce energy consumption.
    LRTV Custom TV
    Deluxe's Unified Delivery Solution

    5|3|16   |     |   (0) comments


    Join Alan Breznick of Light Reading and visit the Deluxe booth at NAB! Here you'll find Deluxe's Unified Delivery Solution, OTT video, virtual reality, HDR, 4K and much more!
    LRTV Interviews
    Verizon Puts Gray Boxes in the Shade

    5|2|16   |   04:33   |   (1) comment


    When it comes to the white box trend, "gray" boxes, which have a slight proprietary twist, don't give service providers and end users the advantages they're seeking, according to Verizon's Vice President of Product and New Business Innovation Shawn Hakl.
    LRTV Custom TV
    Dealing With a Disrupted Video Market

    5|2|16   |     |   (0) comments


    Ericsson's Simon Frost discusses how traditional pay-TV providers can cope with the big changes wrought by the rise of OTT video and IP technology.
    LRTV Custom TV
    The VNF Responsibility of Red Hat

    5|2|16   |     |   (0) comments


    At MWC, Caroline Chappell of Heavy Reading visits the Red Hat booth and sits down with Chris Wright to talk about the responsibility the VNF needs to take on in order to ensure the operators get the carrier-grade performance they expect for their network.
    LRTV Interviews
    AT&T Expert on the Key Pillars of UC

    4|29|16   |   03:58   |   (0) comments


    Vishy Gopalakrishnan, AVP of product marketing at AT&T, talks about the three developments that are making unified communications and collaboration secure and reliable for enterprise users.
    LRTV Documentaries
    LRTV Report: Mobile Core Innovation

    4|28|16   |   25:32   |   (0) comments


    Hear from multiple industry experts from Deutsche Telekom, SK Telecom, Heavy Reading, Huawei, Cisco, Ericsson, Nokia, NEC and many more about developments in the mobile core as operators virtualize their IMS and evolved packet core systems and prepare for a 5G world.
    LRTV Huawei Video Resource Center
    NFV World Congress Highlight

    4|26|16   |     |   (0) comments


    The highlight of the NFV World Congress contains exciting telecom news. Join us for an inside look at Huawei's ICT 2020 plan and its latest collaboration with industry leaders.
    LRTV Interviews
    Unified Comms Finds Its Voice

    4|25|16   |   03:44   |   (0) comments


    Peter Quinlan, VP of UCC Product Management at Tata Communications, talks about the evolution of the unified communications and collaboration services sector and how voice is now a big part of current developments.
    LRTV Documentaries
    So... What Do We Do Now?

    4|25|16   |   03:24   |   (0) comments


    After a long hiatus, Max Dingman, the CEO of a GeeGhiz, returns for a motivational board room pep talk.
    LRTV Documentaries
    NAB 2016 Highlights

    4|21|16   |     |   (0) comments


    Light Reading's Cable/Video Practice Leader Alan Breznick climbs down from the slots to tell us about the latest news in broadcast technology at NAB 2016 in Las Vegas.
    Between the CEOs
    CEO Chat: Deepfield's Craig Labovitz

    4|21|16   |     |   (0) comments


    In this latest installment of the CEO Chat series, Craig Labovitz, co-founder and CEO of Deepfield, sits down with Light Reading's Steve Saunders in Light Reading's New York City office to discuss how Deepfield fits in with the big data trend and more.
    Upcoming Live Events
    May 23, 2016, Austin, TX
    May 23, 2016, Austin Convention Center
    May 24-25, 2016, Austin Convention Center, Austin, TX
    September 13-14, 2016, The Curtis Hotel, Denver, CO
    December 6-8, 2016,
    June 16-18, 2017, Austin Convention Center, Austin, TX
    All Upcoming Live Events
    Infographics
    A new survey conducted by Heavy Reading and TM Forum shows that CSPs around the world see the move to digital operations as a necessary part of their overall virtualization strategies.
    Hot Topics
    Ultra-Broadband Summit, Hong Kong
    Iain Morris, News Editor, 4/27/2016
    Amazon AWS Reports $2.6B Quarterly Revenue, Up a Colossal 64%
    Mitch Wagner, West Coast Bureau Chief, Light Reading, 4/28/2016
    WiCipedia: Woman Cards & Bitch Switches
    Sarah Thomas, Director, Women in Comms, 4/29/2016
    FCC Poised to Re-Regulate Wholesale Access
    Carol Wilson, Editor-at-large, 4/28/2016
    LoRa Alliance Defends Tech Against Sigfox Slur
    Iain Morris, News Editor, 4/28/2016
    Like Us on Facebook
    Twitter Feed
    BETWEEN THE CEOs - Executive Interviews
    Mark Dzuban, President and CEO of SCTE, sits down with Steve Saunders in Light Reading's New York office to discuss the society's Energy 2020 campaign, including its mission to benefit the environment, enable economic benefits and the key challenges facing cable operators wanting to reduce energy consumption.
    In this latest installment of the CEO Chat series, Craig Labovitz, co-founder and CEO of Deepfield, sits down with Light Reading's Steve Saunders in Light Reading's New York City office to discuss how Deepfield fits in with the big data trend and more.
    Animals with Phones
    Sloth Mail Click Here
    Sloth mail -- somehow even slower than snail mail.
    Live Digital Audio

    Of all the tech companies in the Valley, Intel has made the most aggressive commitment to building a diverse and inclusive workplace culture. It's doing so by taking concrete, measurable steps, making a large financial investment and through a commitment to complete transparency about its progress. In this radio show, WiC Director Sarah Thomas will be joined by Shlomit Weiss, Intel's Vice President, Data Center Group, and General Manager of Networking Engineering, who will share with us why Intel is tackling this huge challenge, how and to what effect. She will also discuss her unique experiences leading development of Client SOC development in the past and today leading development of all of the chipmaker's silicon hardware for networking IPs and discrete devices and managing a team of 600 engineers across Israel, Europe and the US.