& cplSiteName &

The 400Gbit/s Demonstrator

Eduard Beier
8/21/2013
100%
0%

A collaboration between research institutions and industrial partners demonstrated at the ISC'13 that 400Gbit/s bandwidth granularity is not only feasible, but already useful today.

For demonstration purposes a number of innovative technologies such as 400Gbit/s DWDM Super Channel, high-speed flash memory and a distributed parallel file system are used in combination.

The project is set up to have real data and realistic applications instead of yet another "hero experiment" with test generators and analyzers. Initial performance tests were performed to check the operational condition of all components working together in the demonstrator. Synthetic data and load was put on the connectivity and IT components and showed overall good operational conditions.

Then two applications were activated on the demonstrator:

  • Climate Research with centralized computing accessing to distributed data

  • Turbine Development with multi-stage processing dynamically both shifting big-data between Munich and Dresden (620km standard fiber) over a single 400Gbit/s wavelength Super Channel.

    From the research community are involved: the German Centre for Aerospace (DLR); the Leibniz Supercomputing Centre (LRZ); the Max Planck Institute for Meteorology (MPI-M) from the Max Planck Society and the computing center of the Max Planck Society in Garching (RZG); the computer center of the Technical University of Freiberg (RZ TUBAF); and the Center for Information Services and High Performance Computing Dresden University of Technology (ZIH).

    Industrial partners are: Alcatel-Lucent (NYSE: ALU), Barracuda Networks Inc. , Bull SA , Cluster Vision, EMC2, IBM Corp. (NYSE: IBM), Mellanox Technologies Ltd. (Nasdaq: MLNX), Deutsche Telekom AG (NYSE: DT), and T-Systems International GmbH .

    The fundamental network structure of the Demonstrator is shown by the following picture:

    Network Structure

    The compute clusters on both sides are clients of the distributed parallel file system which consists of 12 servers with three flash memory boards in each server. Both clusters are able to read and write at 400 Gbit/s on that file system.

    Standard Sandy Bridge servers with at least five PCIe3.0x8 slots and 128GB DRAM are used. The design goal was to create about 18 GBytes of sustained duplex data rate per node (Ethernet: 5GBytes, Memory: 6 GBytes, Fabric: 7 GBytes). The Fabric data rate has not yet been tested, the other rates have been confirmed by tests.

    For performance margin reasons the setup is moderately overbooked between memory and network. Below are the total theoretical performance numbers.

    Theoretical Throughput

    The predecessor project in 2010 (100Gbit/s Testbed Dresden Freiberg) required arrays of about 800 spinning disks on each side to form a 100Gbit/s data path. Because that setup was, for scaling reasons, not possible for 400 Gbit/s, the 2 GBytes I/O throughput flash memory board is one of the enablers of this 400Gbit/s project.

    For the ISC configuration (in the picture below), the following systems were added: a 10GbE Cluster in Dresden; a 2x100GbE link to Freiberg (not connected during ISC); in Garching the SuperMUC at LRZ (one of the fastest supercomputers in Europe) and a 480 GBytes cluster (> 2500 cores) at RZG; and a commercial Cloud Service from T-Systems (not connected during ISC).

    The Big Picture

    The project links to SDN and NFV -- in particular as it is very active in NSI (Network-to-Service-Interface) definition. (For further information, see the link below to additional project information).

    Because the "distributed high speed GPFS" approach is of some universal nature (e.g. for HPC datacenter backup and HPC workload distribution reasons) the setup will be tested for commercial applicability during the post ISC phase. The possibility to use network functions like encryption, firewalling and data compression is definitely a must in a commercial case.

    Network appliances for 40 Gbit/s and 100 Gbit/s are neither available nor would they be affordable in many cases. Therefore we are going to test virtualized network functions on standard server hardware (see picture below); that additional "module," which is based on the same server hardware as the other servers, gets in between each server and the router on each side.

    NFV Module

    Please feel free to ask questions about any technical aspect of the project. The project team, which consists of some of the best specialists on their individual field, will be happy to answer. For additional project information: http://tu-dresden.de/zih/400gigabit.

    The Partners

    — Eduard Beier, process systems engineer, T-Systems International

    (6)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    RolfSperber
    50%
    50%
    RolfSperber,
    User Rank: Light Beer
    9/26/2013 | 10:11:00 AM
    Re: SDN and NFV
    Ray, we are at a very early stage, but abstracting from both hardware (and in consequence IOS) layer and utilizing a common framework (see NSI WG in OGF) and at the same time allow for docking of virtalized network functions will work for multi vendor, multi carrier and multi domain. Still, its a long way to go!
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    9/26/2013 | 7:51:13 AM
    Re: SDN and NFV
    Rolf

    You say it is not restricted to a single domain, but is it applicable in networks that traverse multiple infrastructures run and managed by multiple network operators?
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:43:49 AM
    Re: Is this another route in for NFV?
    Requirements in industry will not be so different from those in R&D. Looking at the plans in the context of Horizon 2020 Puplic Private Partnership is a target of European efforts. Taking into account the cost of infrastructure and the prevailing attitude to pay as little as possible for utilization of infrastructure, more sophisticated multiplexing methods, and this include NFV, are inevitable.

    For network operators this scheme means significantly reduced time to market.
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:36:55 AM
    SDN and NFV
    In this project we will be going a step further. Our plan is to create an environment that allows for creating a virtual network based on the requirements of either applications or  carrier provided network functionality. We will not be restricted to a single domain and connectivity, our target is a network created from building blocks out of a repository. These building blocks can be connectivity with certain quality parameters or virtualized network functions like e.g. firwall functionality, compression, encryption, accelleration.

     

     
    Eddie_
    50%
    50%
    Eddie_,
    User Rank: Blogger
    8/21/2013 | 3:00:47 PM
    Re: Is this another route in for NFV?
    NFV simply scales better than HW based approaches (if NFV tests in September show good results).

    A possible 2013 roadmap for the project:
    • fully SDN controlled network
    • 200GBit/s datapath
    • NFV in the data path 

    a possible 2014 roadmap for the project:
    • scale up to a 1TBit/s data path

    how else could you do that?

     

     

     

     

     
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    8/21/2013 | 1:01:52 PM
    Is this another route in for NFV?
    Interesting that functions virtualization takes the place of appliances that are either too expensive to deploy or have not yet been created... is NFV going to help the R&D sector more than production network operatins in the early years?
    Educational Resources
    sponsor supplied content
    Educational Resources Archive
    More Blogs from Column
    As online video keeps pouring over both fixed and wireless networks, both content and service providers must adapt by adjusting their operations and business strategies.
    The latest Future of Virtualization Index data suggests opex savings from NFV are not materializing as expected – but it doesn't have to be that way, according to Leo Zancani.
    Deutsche Telekom is taking a collaborative approach to the development of NB-IoT applications, explains Dr. Alexander Lautz.
    NFV and mobile edge computing are exciting industry developments but they require hardware evolution just as much as software evolution, argues Chloe Jian Ma.
    As CSPs offer faster wireline and wireless services, they must focus more on monitoring and assuring the customer's quality of experience.
    Light Reading’s Upskill U is a FREE, interactive, online educational resource that delivers must-have education on themes that relate to the overall business transformation taking place in the communications industry.
    NEXT COURSE
    Wednesday, July 27, 1:00PM EDT
    The Changing Face of the Data Center World
    Rodney M. Elder, Senior Solutions Architect, Equinix
    UPCOMING COURSE SCHEDULE
    Wednesday, August 3, 1:00PM EDT
    The Central Office Re-Architected as a Data Center
    Guru Parulkar, Executive Director, Open Networking Research Center, Open Networking Lab
    Wednesday, August 10, 1:00PM EDT
    Telcos & Open Source 101
    Phil Robb, Senior Technical Director, OpenDaylight
    Friday, August 12, 1:00PM EDT
    The Role of Open Source in NFV
    Jim Fagan, Director, Cloud Practice, Telstra
    in association with:
    From The Founder
    The more things change, the more they stay the same for Juniper's next-gen comms solutions, and that's a good thing.
    Flash Poll
    Live Streaming Video
    Charting the CSP's Future
    Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it's going from two industry veterans.
    LRTV Custom TV
    NetScout: Maximizing Enterprise Cloud for Digital Transformation

    7|20|16   |   04:53   |   (0) comments


    Light Reading Editor Mitch Wagner talks to NetScout CMO Jim McNiel about maximizing the benefits of enterprise cloud and digital transformation while minimizing potential pitfalls with a proper monitoring and instrumentation strategy.
    Women in Comms Introduction Videos
    Ciena's VP Offers a Career Crash Course

    7|20|16   |   4:14   |   (0) comments


    How did Ciena's Vice President of Sales, Angela Finn, carve out her career path? Simple, she tells WiC. She stayed true to her company, customers and principles. She shares her advice for women on how to be authentic and credible, as well as for companies that want to make a real change to their culture and practices.
    LRTV Custom TV
    NFV in 2016: Part 2 – Climbing the Virtualization Maturity Curve

    7|19|16   |   06:56   |   (0) comments


    Many of the initial use case implementations are single-vendor and self-contained. The industry is still climbing the virtualization maturity curve, needing further clarity and stability in the NFV infrastructure (NFVi) and greater availability and choice of virtualized network functions (VNFs). Interoperability between NFVis and VNFs from different vendors ...
    Telecom Innovators Video Showcase
    Versa Networks' Kumar Mehta on SD-WAN Managed Services

    7|19|16   |     |   (0) comments


    In Silicon Valley, Steve Saunders sits down with Versa's Kumar Mehta for an interview focused on why service providers are building SD-WAN managed services, and how Versa's telco customers are innovating.
    LRTV Custom TV
    Juniper Networks & The Evolution of NFV

    7|19|16   |   06:01   |   (0) comments


    Senior Juniper Networks executives talk to Light Reading Founder & CEO Steve Saunders about NFV developments and the recent independent evaluation by test lab EANTC of Juniper's Cloud CPE solution.
    LRTV Interviews
    CenturyLink Goes Beyond Managed WiFi

    7|19|16   |     |   (0) comments


    CenturyLink's managed WiFi allows enterprises, such as retailers and resorts, to track guest WiFi usage in order to help them better communicate with customers.
    LRTV Interviews
    AT&T Launches Network Functions on Demand

    7|17|16   |   05:26   |   (0) comments


    Roman Pacewicz, Senior Vice President, Offer Management & Service Integration, AT&T Business Solutions, discusses the operator's launch of its Network Functions on Demand service.
    LRTV Interviews
    Enterprise Pitch for Ciscosson

    7|14|16   |   04:43   |   (0) comments


    After seven months of near silence, Cisco and Ericsson executives publicly discussed details on their extensive partnership. Among the tidbits shared by Martin Zander, VP, group strategy programs, Ericsson, and Doug Webster, VP service provider marketing, Cisco: The partnership was initially launched to serve the service provider market, but is already gaining ...
    Wagner’s Ring
    Cisco Faces Up to Hypercloud Threat

    7|13|16   |   02:42   |   (0) comments


    Facebook, Amazon and Google mostly don't buy branded technology for their networks – they build their own. That's a threat to Cisco – and its competitors too – which face potentially dwindling demand for their product. Is Cisco up to the challenge? Light Reading went to the annual Cisco Live conference to find out.
    LRTV Huawei Video Resource Center
    Building a Better Connected Russia

    7|13|16   |     |   (0) comments


    At UBBS World Tour 2016, Alla Shabelnikova of Ovum shares the findings of a white paper outlining the challenges and opportunities of broadband rollout in Russia.
    LRTV Huawei Video Resource Center
    The Global Video Business

    7|13|16   |     |   (0) comments


    At UBBS World Tour 2016, Roger Feng of Huawei shares insights on the future of video business.
    LRTV Huawei Video Resource Center
    UBBS World Tour Moscow Highlights

    7|13|16   |     |   (0) comments


    At UBBS World Tour 2016 at Moscow, Huawei showcases its outstanding progress in video technology.
    Upcoming Live Events
    September 13-14, 2016, The Curtis Hotel, Denver, CO
    September 27, 2016, Philadelphia, PA
    November 3, 2016, The Montcalm Marble Arch, London
    November 30, 2016, The Westin Times Square, New York City
    December 6-8, 2016,
    May 16-17, 2017, Austin Convention Center, Austin, TX
    All Upcoming Live Events
    Infographics
    Five of the Top 10 most targeted countries in Check Point Software Technologies' global Malware & Threat Index for Q1 2016 are in Africa.
    Hot Topics
    SoftBank Muscles In on ARM in $32B Deal
    Iain Morris, News Editor, 7/18/2016
    Ericsson 'Doubles' Savings Goal as Sales Slump
    Iain Morris, News Editor, 7/19/2016
    Kevin Lo's Move to Facebook: Sign of Things to Come?
    Patrick Donegan, Chief Analyst, Heavy Reading, 7/20/2016
    Verizon's Next With VNFs
    Carol Wilson, Editor-at-large, 7/21/2016
    Facebook Gets Its Drone On
    Ray Le Maistre, Editor-in-chief, 7/22/2016
    Like Us on Facebook
    Twitter Feed
    BETWEEN THE CEOs - Executive Interviews
    There's no question that, come 2020, 5G technology will turn the world's conception of what mobile networking is on its head. Within the world of 5G development, Dr. ...
    I've enjoyed interviewing many interesting people since I rejoined Light Reading, but William A. "Bill" Owens certainly takes the biscuit, as we say where I come from.
    Live Digital Audio

    Our world has evolved through innovation from the Industrial Revolution of the 1740s to the information age, and it is now entering the Fourth Industrial Revolution, driven by technology. Technology is driving a paradigm shift in the way digital solutions deliver a connected world, changing the way we live, communicate and provide solutions. It can have a powerful impact on how we tackle some of the world’s most pressing problems. In this radio show, Caroline Dowling, President of Communications Infrastructure & Enterprise Computing at Flex, will join Women in Comms Director Sarah Thomas to discuss the impact technology has on society and how it can be a game-changer across the globe; improving lives and creating a smarter world. Dowling, a Cork, Ireland, native and graduate of Harvard Business School's Advanced Management Program, will also discuss her experience managing an international team focused on innovation in an age of high-speed change.