& cplSiteName &

The 400Gbit/s Demonstrator

Eduard Beier
8/21/2013
100%
0%

A collaboration between research institutions and industrial partners demonstrated at the ISC'13 that 400Gbit/s bandwidth granularity is not only feasible, but already useful today.

For demonstration purposes a number of innovative technologies such as 400Gbit/s DWDM Super Channel, high-speed flash memory and a distributed parallel file system are used in combination.

The project is set up to have real data and realistic applications instead of yet another "hero experiment" with test generators and analyzers. Initial performance tests were performed to check the operational condition of all components working together in the demonstrator. Synthetic data and load was put on the connectivity and IT components and showed overall good operational conditions.

Then two applications were activated on the demonstrator:

  • Climate Research with centralized computing accessing to distributed data

  • Turbine Development with multi-stage processing dynamically both shifting big-data between Munich and Dresden (620km standard fiber) over a single 400Gbit/s wavelength Super Channel.

    From the research community are involved: the German Centre for Aerospace (DLR); the Leibniz Supercomputing Centre (LRZ); the Max Planck Institute for Meteorology (MPI-M) from the Max Planck Society and the computing center of the Max Planck Society in Garching (RZG); the computer center of the Technical University of Freiberg (RZ TUBAF); and the Center for Information Services and High Performance Computing Dresden University of Technology (ZIH).

    Industrial partners are: Alcatel-Lucent (NYSE: ALU), Barracuda Networks Inc. , Bull SA , Cluster Vision, EMC2, IBM Corp. (NYSE: IBM), Mellanox Technologies Ltd. (Nasdaq: MLNX), Deutsche Telekom AG (NYSE: DT), and T-Systems International GmbH .

    The fundamental network structure of the Demonstrator is shown by the following picture:

    Network Structure

    The compute clusters on both sides are clients of the distributed parallel file system which consists of 12 servers with three flash memory boards in each server. Both clusters are able to read and write at 400 Gbit/s on that file system.

    Standard Sandy Bridge servers with at least five PCIe3.0x8 slots and 128GB DRAM are used. The design goal was to create about 18 GBytes of sustained duplex data rate per node (Ethernet: 5GBytes, Memory: 6 GBytes, Fabric: 7 GBytes). The Fabric data rate has not yet been tested, the other rates have been confirmed by tests.

    For performance margin reasons the setup is moderately overbooked between memory and network. Below are the total theoretical performance numbers.

    Theoretical Throughput

    The predecessor project in 2010 (100Gbit/s Testbed Dresden Freiberg) required arrays of about 800 spinning disks on each side to form a 100Gbit/s data path. Because that setup was, for scaling reasons, not possible for 400 Gbit/s, the 2 GBytes I/O throughput flash memory board is one of the enablers of this 400Gbit/s project.

    For the ISC configuration (in the picture below), the following systems were added: a 10GbE Cluster in Dresden; a 2x100GbE link to Freiberg (not connected during ISC); in Garching the SuperMUC at LRZ (one of the fastest supercomputers in Europe) and a 480 GBytes cluster (> 2500 cores) at RZG; and a commercial Cloud Service from T-Systems (not connected during ISC).

    The Big Picture

    The project links to SDN and NFV -- in particular as it is very active in NSI (Network-to-Service-Interface) definition. (For further information, see the link below to additional project information).

    Because the "distributed high speed GPFS" approach is of some universal nature (e.g. for HPC datacenter backup and HPC workload distribution reasons) the setup will be tested for commercial applicability during the post ISC phase. The possibility to use network functions like encryption, firewalling and data compression is definitely a must in a commercial case.

    Network appliances for 40 Gbit/s and 100 Gbit/s are neither available nor would they be affordable in many cases. Therefore we are going to test virtualized network functions on standard server hardware (see picture below); that additional "module," which is based on the same server hardware as the other servers, gets in between each server and the router on each side.

    NFV Module

    Please feel free to ask questions about any technical aspect of the project. The project team, which consists of some of the best specialists on their individual field, will be happy to answer. For additional project information: http://tu-dresden.de/zih/400gigabit.

    The Partners

    — Eduard Beier, process systems engineer, T-Systems International

    (6)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    RolfSperber
    50%
    50%
    RolfSperber,
    User Rank: Light Beer
    9/26/2013 | 10:11:00 AM
    Re: SDN and NFV
    Ray, we are at a very early stage, but abstracting from both hardware (and in consequence IOS) layer and utilizing a common framework (see NSI WG in OGF) and at the same time allow for docking of virtalized network functions will work for multi vendor, multi carrier and multi domain. Still, its a long way to go!
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    9/26/2013 | 7:51:13 AM
    Re: SDN and NFV
    Rolf

    You say it is not restricted to a single domain, but is it applicable in networks that traverse multiple infrastructures run and managed by multiple network operators?
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:43:49 AM
    Re: Is this another route in for NFV?
    Requirements in industry will not be so different from those in R&D. Looking at the plans in the context of Horizon 2020 Puplic Private Partnership is a target of European efforts. Taking into account the cost of infrastructure and the prevailing attitude to pay as little as possible for utilization of infrastructure, more sophisticated multiplexing methods, and this include NFV, are inevitable.

    For network operators this scheme means significantly reduced time to market.
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:36:55 AM
    SDN and NFV
    In this project we will be going a step further. Our plan is to create an environment that allows for creating a virtual network based on the requirements of either applications or  carrier provided network functionality. We will not be restricted to a single domain and connectivity, our target is a network created from building blocks out of a repository. These building blocks can be connectivity with certain quality parameters or virtualized network functions like e.g. firwall functionality, compression, encryption, accelleration.

     

     
    Eddie_
    50%
    50%
    Eddie_,
    User Rank: Blogger
    8/21/2013 | 3:00:47 PM
    Re: Is this another route in for NFV?
    NFV simply scales better than HW based approaches (if NFV tests in September show good results).

    A possible 2013 roadmap for the project:
    • fully SDN controlled network
    • 200GBit/s datapath
    • NFV in the data path 

    a possible 2014 roadmap for the project:
    • scale up to a 1TBit/s data path

    how else could you do that?

     

     

     

     

     
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    8/21/2013 | 1:01:52 PM
    Is this another route in for NFV?
    Interesting that functions virtualization takes the place of appliances that are either too expensive to deploy or have not yet been created... is NFV going to help the R&D sector more than production network operatins in the early years?
    Educational Resources
    sponsor supplied content
    Educational Resources Archive
    More Blogs from Column
    5G could ride the traditional wireless hype cycle, or – quite possibly – break the chain, suggests Nokia's North American CTO.
    NFV feels like it's going nowhere fast, suggests Napatech's Dan Joe Barry.
    The NG-PON2 standard is the key to the future of ultra-broadband, believes Calix's Alan DiCicco.
    Amazing new 5G innovations are coming below 6GHz -- it's not just about mmWave.
    5G is about more than just faster mobile broadband, notes Tom Sawanobori, SVP and chief technology officer at the CTIA.
    Light Reading’s Upskill U is a FREE, interactive, online educational resource that delivers must-have education on themes that relate to the overall business transformation taking place in the communications industry.
    NEXT COURSE
    Wednesday, October 5, 1:00PM EDT
    Gigabit & Smart Cities
    Joe Kochan, COO & Co-Founder, US Ignite
    UPCOMING COURSE SCHEDULE
    Friday, October 7, 1:00PM EDT
    Gigabit & DOCSIS 3.1
    Ty Pearman, Director, Access Architecture, Comcast
    Wednesday, October 19, 1:00PM EDT
    Securing a Virtual World
    Rita Marty, Executive Director, Mobility and Cloud Security, Chief Security Office, AT&T
    Friday, October 21, 1:00PM EDT
    Security: Evolving the Data Center
    Rasool Kareem Irfan, Head, Telecom & Infrastructure Security Practice, Tata Communications Transformation Services Ltd (TCTS)
    in association with:
    From The Founder
    Light Reading today starts a new voyage as part of a larger Enterprise.
    Flash Poll
    Live Streaming Video
    Charting the CSP's Future
    Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it's going from two industry veterans.
    LRTV Documentaries
    From Philly, With Love

    9|30|16   |     |   (5) comments


    Join Alan Breznick, cable's answer to the Italian Stallion, as he runs through the highlights of SCTE Cable-Tec Expo, lumbers along in Rocky Balboa's footsteps and searches for the perfect Philadelphia cheesesteak.
    LRTV Interviews
    CenturyLink: SD-WAN Customers Looking for Value Not Cost Savings

    9|30|16   |   5:31   |   (0) comments


    At NFV & Carrier SDN in Denver, CenturyLink's Eric Nowak told Light Reading that when customers launch SD-WAN, they aren't necessary looking to save money, but instead they are looking for more value from what they're spending. He also shared some unique case studies and lessons learned from launching SD-WAN services.
    LRTV Custom TV
    Flexible Deployment Approaches for the Gigabit Services Evolution

    9|29|16   |     |   (0) comments


    For many operators, the gigabit evolution begins with the shift from DOCSIS 3.0 to DOCSIS 3.1. But that move represents a change not only in the protocol itself, but in the approach to architecting their entire DOCSIS delivery chain -- from the headend to the outside plant and home gateway components.

    Jonathan Ruff, senior director of global technical ...

    LRTV Interviews
    Level 3 VP: Enterprises Need More for Less

    9|29|16   |   05:27   |   (0) comments


    Andrew Dugan, Level 3 group vice president of global technology and IT, says enterprises need more bandwidth and they need it faster and with greater security, but they want to spend less, if possible. They are looking to carriers to reduce their network complexity and help protect them from cyberattacks as well.
    LRTV Interviews
    CenturyLink: SDN/NFV Pose New Interconnection Possibilities

    9|28|16   |   04:37   |   (0) comments


    Network operators should develop new APIs and business processes for reselling virtual assets to each other, says CenturyLink's Bill Walker. That will enable them to build digital business portfolios that help them avoid becoming commodity transport providers.
    LRTV Interviews
    Level 3: Overcoming Terror of Being Supplier, Integrator & Developer

    9|28|16   |     |   (0) comments


    At Light Reading's NFV & Carrier SDN event in Denver, Travis Ewert of Level 3 Communications said there is terror in becoming supplier, integrator and developer, but it can be overcome and be cost effective.
    LRTV Custom TV
    Introducing IoT World News

    9|27|16   |   01:43   |   (0) comments


    Self-driving cars, medical sensors, smart cities... and refrigerators. In order to address the huge scope of IoT, KNect365 has created a unique online community that will help businesses to understand and monetize the opportunities that live within the IoT market. We look forward to welcoming you to IoT World News -- your gateway to a better connected future.
    LRTV Interviews
    AT&T: Reusable Functions Next NFV Key

    9|27|16   |   06:03   |   (0) comments


    The next generation of NFV has to break functions down into reusable software chunks, making everything much more cloud-like.
    LRTV Interviews
    Masergy on Security: Attackers Gaining Upper Hand

    9|27|16   |   5:10   |   (2) comments


    At Light Reading's NFV & Carrier SDN event in Denver, Ray Watson, vice president of Global Technology at Masergy, says that because of the growth in virtualization, the threat landscape is shifting in favor of the attackers. As a result, service providers need to think beyond just defending the perimeter and take a more holistic approach to security.
    LRTV Interviews
    Verizon Takes Next Step on Biz Virtualization Journey

    9|26|16   |   4:38   |   (2) comments


    At September's NFV & Carrier SDN event in Denver, Light Reading sat down with Victoria Lonker, director of Product and New Business Innovation at Verizon, to chat about where the carrier is with delivering virtualized services to business customers.
    LRTV Interviews
    Global Services: The $40B Face-Off

    9|26|16   |   05:53   |   (1) comment


    More service providers than ever before are battling it out to win a slice of what is now a $40 billion global communications services pie, explains Ovum Principal Analyst David Molony.
    LRTV Documentaries
    MEC Congress: The Key Takeaways

    9|22|16   |   03:25   |   (3) comments


    Three key takeaways from the Mobile Edge Computing (MEC) Congress in Munich, Germany.
    Upcoming Live Events
    November 3, 2016, The Montcalm Marble Arch, London
    November 30, 2016, The Westin Times Square, New York City
    December 1, 2016, The Westin Times Square, New York, NY
    December 6-8, 2016, The Westin Excelsior, Rome
    May 16-17, 2017, Austin Convention Center, Austin, TX
    All Upcoming Live Events
    Infographics
    Hot Topics
    Eurobites: Telefónica Taps Juniper for Network Security
    Paul Rainford, Assistant Editor, Europe, 9/26/2016
    AT&T CEO Backs Black Lives Matter
    Dan Jones, Mobile Editor, 9/30/2016
    Powell Kills the Cable Show
    Mari Silbey, Senior Editor, Cable/Video, 9/29/2016
    Telstra Sees Quadrupled Data Capacity by 2020
    Carol Wilson, Editor-at-large, 9/28/2016
    From Philly, With Love
    Alan Breznick, Cable/Video Practice Leader, Light Reading, 9/30/2016
    Like Us on Facebook
    Twitter Feed
    BETWEEN THE CEOs - Executive Interviews
    Light Reading CEO Steve Saunders and UXP Systems CEO Gemini Waghmare discuss the strategic importance of digital identity for operators in the midst of transformation.
    Join us for an in-depth interview between Steve Saunders of Light Reading and Alexis Black Bjorlin of Intel as they discuss the release of the company's Silicon Photonics platform, its performance, long-term prospects, customer expectations and much more.
    Animals with Phones
    There's Nothing Like Missing a Full Minute of Pokémon Go Click Here
    Live Digital Audio

    A vital part of increasing the number of women in comms is transforming the ways companies can support and empower women. While progressive company policies that support both men and women in achieving work-life balance are a step in the right direction, creating a company culture that supports those policies can at times be more challenging.

    During this show, we'll talk to Lynn Comp, Senior Director of Industry and Sales Enabling (ISE) in the Network Platforms Group at Intel, about why those challenges exist and how companies can overcome them. She'll provide insight into how Intel has worked to create a culture that supports work-life balance, and provide steps and guidance for other companies wishing to do the same. We will also leave plenty of time to get your questions answered live on the air.