& cplSiteName &

The 400Gbit/s Demonstrator

Eduard Beier
8/21/2013
100%
0%

A collaboration between research institutions and industrial partners demonstrated at the ISC'13 that 400Gbit/s bandwidth granularity is not only feasible, but already useful today.

For demonstration purposes a number of innovative technologies such as 400Gbit/s DWDM Super Channel, high-speed flash memory and a distributed parallel file system are used in combination.

The project is set up to have real data and realistic applications instead of yet another "hero experiment" with test generators and analyzers. Initial performance tests were performed to check the operational condition of all components working together in the demonstrator. Synthetic data and load was put on the connectivity and IT components and showed overall good operational conditions.

Then two applications were activated on the demonstrator:

  • Climate Research with centralized computing accessing to distributed data

  • Turbine Development with multi-stage processing dynamically both shifting big-data between Munich and Dresden (620km standard fiber) over a single 400Gbit/s wavelength Super Channel.

    From the research community are involved: the German Centre for Aerospace (DLR); the Leibniz Supercomputing Centre (LRZ); the Max Planck Institute for Meteorology (MPI-M) from the Max Planck Society and the computing center of the Max Planck Society in Garching (RZG); the computer center of the Technical University of Freiberg (RZ TUBAF); and the Center for Information Services and High Performance Computing Dresden University of Technology (ZIH).

    Industrial partners are: Alcatel-Lucent (NYSE: ALU), Barracuda Networks Inc. , Bull SA , Cluster Vision, EMC2, IBM Corp. (NYSE: IBM), Mellanox Technologies Ltd. (Nasdaq: MLNX), Deutsche Telekom AG (NYSE: DT), and T-Systems International GmbH .

    The fundamental network structure of the Demonstrator is shown by the following picture:

    Network Structure

    The compute clusters on both sides are clients of the distributed parallel file system which consists of 12 servers with three flash memory boards in each server. Both clusters are able to read and write at 400 Gbit/s on that file system.

    Standard Sandy Bridge servers with at least five PCIe3.0x8 slots and 128GB DRAM are used. The design goal was to create about 18 GBytes of sustained duplex data rate per node (Ethernet: 5GBytes, Memory: 6 GBytes, Fabric: 7 GBytes). The Fabric data rate has not yet been tested, the other rates have been confirmed by tests.

    For performance margin reasons the setup is moderately overbooked between memory and network. Below are the total theoretical performance numbers.

    Theoretical Throughput

    The predecessor project in 2010 (100Gbit/s Testbed Dresden Freiberg) required arrays of about 800 spinning disks on each side to form a 100Gbit/s data path. Because that setup was, for scaling reasons, not possible for 400 Gbit/s, the 2 GBytes I/O throughput flash memory board is one of the enablers of this 400Gbit/s project.

    For the ISC configuration (in the picture below), the following systems were added: a 10GbE Cluster in Dresden; a 2x100GbE link to Freiberg (not connected during ISC); in Garching the SuperMUC at LRZ (one of the fastest supercomputers in Europe) and a 480 GBytes cluster (> 2500 cores) at RZG; and a commercial Cloud Service from T-Systems (not connected during ISC).

    The Big Picture

    The project links to SDN and NFV -- in particular as it is very active in NSI (Network-to-Service-Interface) definition. (For further information, see the link below to additional project information).

    Because the "distributed high speed GPFS" approach is of some universal nature (e.g. for HPC datacenter backup and HPC workload distribution reasons) the setup will be tested for commercial applicability during the post ISC phase. The possibility to use network functions like encryption, firewalling and data compression is definitely a must in a commercial case.

    Network appliances for 40 Gbit/s and 100 Gbit/s are neither available nor would they be affordable in many cases. Therefore we are going to test virtualized network functions on standard server hardware (see picture below); that additional "module," which is based on the same server hardware as the other servers, gets in between each server and the router on each side.

    NFV Module

    Please feel free to ask questions about any technical aspect of the project. The project team, which consists of some of the best specialists on their individual field, will be happy to answer. For additional project information: http://tu-dresden.de/zih/400gigabit.

    The Partners

    — Eduard Beier, process systems engineer, T-Systems International

    (6)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    RolfSperber
    50%
    50%
    RolfSperber,
    User Rank: Light Beer
    9/26/2013 | 10:11:00 AM
    Re: SDN and NFV
    Ray, we are at a very early stage, but abstracting from both hardware (and in consequence IOS) layer and utilizing a common framework (see NSI WG in OGF) and at the same time allow for docking of virtalized network functions will work for multi vendor, multi carrier and multi domain. Still, its a long way to go!
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    9/26/2013 | 7:51:13 AM
    Re: SDN and NFV
    Rolf

    You say it is not restricted to a single domain, but is it applicable in networks that traverse multiple infrastructures run and managed by multiple network operators?
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:43:49 AM
    Re: Is this another route in for NFV?
    Requirements in industry will not be so different from those in R&D. Looking at the plans in the context of Horizon 2020 Puplic Private Partnership is a target of European efforts. Taking into account the cost of infrastructure and the prevailing attitude to pay as little as possible for utilization of infrastructure, more sophisticated multiplexing methods, and this include NFV, are inevitable.

    For network operators this scheme means significantly reduced time to market.
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:36:55 AM
    SDN and NFV
    In this project we will be going a step further. Our plan is to create an environment that allows for creating a virtual network based on the requirements of either applications or  carrier provided network functionality. We will not be restricted to a single domain and connectivity, our target is a network created from building blocks out of a repository. These building blocks can be connectivity with certain quality parameters or virtualized network functions like e.g. firwall functionality, compression, encryption, accelleration.

     

     
    Eddie_
    50%
    50%
    Eddie_,
    User Rank: Blogger
    8/21/2013 | 3:00:47 PM
    Re: Is this another route in for NFV?
    NFV simply scales better than HW based approaches (if NFV tests in September show good results).

    A possible 2013 roadmap for the project:
    • fully SDN controlled network
    • 200GBit/s datapath
    • NFV in the data path 

    a possible 2014 roadmap for the project:
    • scale up to a 1TBit/s data path

    how else could you do that?

     

     

     

     

     
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    8/21/2013 | 1:01:52 PM
    Is this another route in for NFV?
    Interesting that functions virtualization takes the place of appliances that are either too expensive to deploy or have not yet been created... is NFV going to help the R&D sector more than production network operatins in the early years?
    Educational Resources
    sponsor supplied content
    Educational Resources Archive
    More Blogs from Column
    In the second part of a two-part series, Paul Robinson explains why the move to live IP production is an opportunity to significantly change the viewing experience, and more importantly, open up new revenue and business opportunities.
    One thing is certain: the coming technology changes will affect everyone to varying degrees, including content providers, service providers and equipment makers.
    It's time for our industry to call BS on the US government's position on Chinese telco manufacturers.
    There are important considerations for companies to keep in mind when moving Domain Name System (DNS) infrastructure to an NFV implementation.
    The lies, damned lies and statistics of real open access network operations.
    From The Founder
    Light Reading sits down at CES with the head of Cisco's service provider video business, Conrad Clemson, to discuss how NFV and cloud security relate to video, the challenge of managing 4K/8K traffic, the global expansion of Netflix and virtual reality.
    Flash Poll
    Live Streaming Video
    CLOUD / MANAGED SERVICES: Prepping Ethernet for the Cloud
    Moderator: Ray LeMaistre Panelists: Jeremy Bye, Leonard Sheahan
    LRTV Documentaries
    All Change in Video

    2|11|16   |   33:12   |   (1) comment


    At this moderated panel at 2020 Vision in Dublin, Alan Breznick, Cable/Video Practice Leader of Light Reading, sits down with Jeff Finkelstein, director of network architecture at Cox Communications, to discuss the rapidly changing video market.
    LRTV Custom TV
    Hosting in Ireland, Past & Present

    2|10|16   |   16:07   |   (0) comments


    Garry Connolly, president of Host in Ireland, presents the keynote at Light Reading's 2020 Vision Executive Summit in Dublin.
    Women in Comms Introduction Videos
    What's Hot in Mobile Commerce?

    2|10|16   |   12:18   |   (1) comment


    Claire Maslen, financial services relationship manager at the GSMA, talks about the development of the digital commerce sector and the types of relationships that mobile operators are developing to further their m-commerce strategies.
    LRTV Documentaries
    EANTC Tests Nokia IP Routing & Mobile Gateway VNFs for Real World Deployment

    2|9|16   |   5:08   |   (1) comment


    Nokia obtained validation of its virtualized router and virtualized mobile gateway capabilities through rigorous testing performed by EANTC. The results set a new industry benchmark for outstanding performance, scalability, resiliency and manageability. Nokia VNFs are ready for telco cloud deployment, so that service providers can accelerate mobile, business and ...
    Between the CEOs
    CEO Chat With Level 3's Jack Waters

    2|8|16   |   26:15   |   (1) comment


    Light Reading CEO and founder Steve Saunders sits down with Level 3 Communications' CTO Jack Waters to discuss hot topics like virtualization, 4K and the future of telecom...
    LRTV Custom TV
    The Composable Telco

    2|8|16   |   24:46   |   (0) comments


    Heavy Reading's Principal Analyst Caroline Chappell presents the keynote at Light Reading's 2020 Vision Executive Summit in Dublin.
    LRTV Custom TV
    Join Us at the Digital Operations Transformation Summit

    2|4|16   |   03:52   |   (0) comments


    The Digital Operations Transformation Summit on February 21, 2016 at the Crowne Plaza Barcelona Fira Centre will bring together 50 senior executives to engage in a unique debate on the opportunities and challenges presented by the transformative evolving digital landscape. RSVP now at events@lightreading.com.
    LRTV Custom TV
    Making the Test: ADVA Ensemble Connector vs. Open vSwitch

    2|4|16   |   01:28   |   (0) comments


    Light Reading, in partnership with EANTC, recently tested ADVA's Ensemble Connector, which replaces open vSwitch and offers carrier-grade capability and interoperability. The test results strengthen ADVA's credibility as a provider in the virtualization space.
    LRTV Custom TV
    Bridging the Gap Between PoCs & Deployment in NFV

    2|4|16   |   31:50   |   (0) comments


    Charlie Ashton of Wind River presents the keynote at Light Reading's 2020 Vision executive summit in Dublin.
    Between the CEOs
    CEO Chat With Mike Aquino

    2|3|16   |   17:34   |   (0) comments


    The former CEO of Overture Networks, Mike Aquino, discusses why truly open virtualization solutions provide service providers with the greatest choice.
    Shades of Ray
    MWC: Buckle Up for 5G & the IIoT

    2|2|16   |   02:28   |   (0) comments


    This year's Mobile World Congress looks set to be a 5G land grab and a chance to get down and dirty with the Industrial Internet of Things (IIoT) – but what will the 5G discussions actually be about?
    LRTV Custom TV
    Case Study: Building China's Next-Gen TV Networks

    2|2|16   |   5:01   |   (0) comments


    With over 2 billion viewers worldwide, Shenzhen Media Group is one of China's largest content producers. By partnering with Huawei and Sobey, SZMG was able to modernize media operations with the Converged News Center, a production studio that is a model for next-generation workflows.
    Upcoming Live Events
    March 10, 2016, The Cable Center, Denver, CO
    April 5, 2016, The Ritz Carlton, Charlotte, NC
    May 23, 2016, Austin, TX
    May 24-25, 2016, Austin Convention Center, Austin, TX
    All Upcoming Live Events
    Hot Topics
    Google's 5G Radio Ambitions Are Expanding
    Dan Jones, Mobile Editor, 2/5/2016
    Cincinnati Bell Joins Weight Watchers Club
    Alan Breznick, Cable/Video Practice Leader, 2/5/2016
    Yahoo & Verizon Sitting in a Tree...
    Brian Santo, Senior editor, Test & Measurement / Components, Light Reading, 2/8/2016
    Vodafone: Flexible Work Policies Boost Profits
    Sarah Thomas, Editorial Operations Director, 2/8/2016
    It's Time to Integrate OTT Video
    Alan Breznick, Cable/Video Practice Leader, 2/8/2016
    Like Us on Facebook
    Twitter Feed
    Webinar Archive
    BETWEEN THE CEOs - Executive Interviews
    Light Reading CEO and founder Steve Saunders sits down with Level 3 Communications' CTO Jack Waters to discuss hot topics like virtualization, 4K and the future of telecom...
    The former CEO of Overture Networks, Mike Aquino, discusses why truly open virtualization solutions provide service providers with the greatest choice.
    Animals with Phones
    Retro Is the Way to Be Click Here
    Some animals are too cute for smartphones.
    Live Digital Audio

    Broadband speeds are ramping up across Europe as the continent, at its own pace, follows North America towards a gigabit society. But there are many steps to take on the road to gigabit broadband availability and a number of technology options that can meet the various requirements of Europe’s high-speed fixed broadband network operators. During this radio show we will look at some of the catalysts for broadband network investments and examine the menu of technology options on offer, including vectoring and G.fast for copper plant evolution and the various deployment possibilities for FTTH/B.