& cplSiteName &

The 400Gbit/s Demonstrator

Eduard Beier
8/21/2013
100%
0%

A collaboration between research institutions and industrial partners demonstrated at the ISC'13 that 400Gbit/s bandwidth granularity is not only feasible, but already useful today.

For demonstration purposes a number of innovative technologies such as 400Gbit/s DWDM Super Channel, high-speed flash memory and a distributed parallel file system are used in combination.

The project is set up to have real data and realistic applications instead of yet another "hero experiment" with test generators and analyzers. Initial performance tests were performed to check the operational condition of all components working together in the demonstrator. Synthetic data and load was put on the connectivity and IT components and showed overall good operational conditions.

Then two applications were activated on the demonstrator:

  • Climate Research with centralized computing accessing to distributed data

  • Turbine Development with multi-stage processing dynamically both shifting big-data between Munich and Dresden (620km standard fiber) over a single 400Gbit/s wavelength Super Channel.

    From the research community are involved: the German Centre for Aerospace (DLR); the Leibniz Supercomputing Centre (LRZ); the Max Planck Institute for Meteorology (MPI-M) from the Max Planck Society and the computing center of the Max Planck Society in Garching (RZG); the computer center of the Technical University of Freiberg (RZ TUBAF); and the Center for Information Services and High Performance Computing Dresden University of Technology (ZIH).

    Industrial partners are: Alcatel-Lucent (NYSE: ALU), Barracuda Networks Inc. , Bull SA , Cluster Vision, EMC2, IBM Corp. (NYSE: IBM), Mellanox Technologies Ltd. (Nasdaq: MLNX), Deutsche Telekom AG (NYSE: DT), and T-Systems International GmbH .

    The fundamental network structure of the Demonstrator is shown by the following picture:

    Network Structure

    The compute clusters on both sides are clients of the distributed parallel file system which consists of 12 servers with three flash memory boards in each server. Both clusters are able to read and write at 400 Gbit/s on that file system.

    Standard Sandy Bridge servers with at least five PCIe3.0x8 slots and 128GB DRAM are used. The design goal was to create about 18 GBytes of sustained duplex data rate per node (Ethernet: 5GBytes, Memory: 6 GBytes, Fabric: 7 GBytes). The Fabric data rate has not yet been tested, the other rates have been confirmed by tests.

    For performance margin reasons the setup is moderately overbooked between memory and network. Below are the total theoretical performance numbers.

    Theoretical Throughput

    The predecessor project in 2010 (100Gbit/s Testbed Dresden Freiberg) required arrays of about 800 spinning disks on each side to form a 100Gbit/s data path. Because that setup was, for scaling reasons, not possible for 400 Gbit/s, the 2 GBytes I/O throughput flash memory board is one of the enablers of this 400Gbit/s project.

    For the ISC configuration (in the picture below), the following systems were added: a 10GbE Cluster in Dresden; a 2x100GbE link to Freiberg (not connected during ISC); in Garching the SuperMUC at LRZ (one of the fastest supercomputers in Europe) and a 480 GBytes cluster (> 2500 cores) at RZG; and a commercial Cloud Service from T-Systems (not connected during ISC).

    The Big Picture

    The project links to SDN and NFV -- in particular as it is very active in NSI (Network-to-Service-Interface) definition. (For further information, see the link below to additional project information).

    Because the "distributed high speed GPFS" approach is of some universal nature (e.g. for HPC datacenter backup and HPC workload distribution reasons) the setup will be tested for commercial applicability during the post ISC phase. The possibility to use network functions like encryption, firewalling and data compression is definitely a must in a commercial case.

    Network appliances for 40 Gbit/s and 100 Gbit/s are neither available nor would they be affordable in many cases. Therefore we are going to test virtualized network functions on standard server hardware (see picture below); that additional "module," which is based on the same server hardware as the other servers, gets in between each server and the router on each side.

    NFV Module

    Please feel free to ask questions about any technical aspect of the project. The project team, which consists of some of the best specialists on their individual field, will be happy to answer. For additional project information: http://tu-dresden.de/zih/400gigabit.

    The Partners

    ó Eduard Beier, process systems engineer, T-Systems International

    (6)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    RolfSperber
    50%
    50%
    RolfSperber,
    User Rank: Light Beer
    9/26/2013 | 10:11:00 AM
    Re: SDN and NFV
    Ray, we are at a very early stage, but abstracting from both hardware (and in consequence IOS) layer and utilizing a common framework (see NSI WG in OGF) and at the same time allow for docking of virtalized network functions will work for multi vendor, multi carrier and multi domain. Still, its a long way to go!
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    9/26/2013 | 7:51:13 AM
    Re: SDN and NFV
    Rolf

    You say it is not restricted to a single domain, but is it applicable in networks that traverse multiple infrastructures run and managed by multiple network operators?
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:43:49 AM
    Re: Is this another route in for NFV?
    Requirements in industry will not be so different from those in R&D. Looking at the plans in the context of Horizon 2020 Puplic Private Partnership is a target of European efforts. Taking into account the cost of infrastructure and the prevailing attitude to pay as little as possible for utilization of infrastructure, more sophisticated multiplexing methods, and this include NFV, are inevitable.

    For network operators this scheme means significantly reduced time to market.
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:36:55 AM
    SDN and NFV
    In this project we will be going a step further. Our plan is to create an environment that allows for creating a virtual network based on the requirements of either applications or  carrier provided network functionality. We will not be restricted to a single domain and connectivity, our target is a network created from building blocks out of a repository. These building blocks can be connectivity with certain quality parameters or virtualized network functions like e.g. firwall functionality, compression, encryption, accelleration.

     

     
    Eddie_
    50%
    50%
    Eddie_,
    User Rank: Blogger
    8/21/2013 | 3:00:47 PM
    Re: Is this another route in for NFV?
    NFV simply scales better than HW based approaches (if NFV tests in September show good results).

    A possible 2013 roadmap for the project:
    • fully SDN controlled network
    • 200GBit/s datapath
    • NFV in the data path 

    a possible 2014 roadmap for the project:
    • scale up to a 1TBit/s data path

    how else could you do that?

     

     

     

     

     
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    8/21/2013 | 1:01:52 PM
    Is this another route in for NFV?
    Interesting that functions virtualization takes the place of appliances that are either too expensive to deploy or have not yet been created... is NFV going to help the R&D sector more than production network operatins in the early years?
    Educational Resources
    sponsor supplied content
    Educational Resources Archive
    More Blogs from Column
    Spectrum sharing is becoming a bigger issue as the 5G radio specification evolves.
    Addressing current and future app demands while laying the foundation for mobile's next big network transition.
    Broadcasters can no longer rely on pulling audiences to the TV screen; they need to pursue their audiences on digital, wherever they are.
    Why advanced data analytics are the future for streaming video services.
    5G could ride the traditional wireless hype cycle, or – quite possibly – break the chain, suggests Nokia's North American CTO.
    Light Readingís Upskill U is a FREE, interactive, online educational resource that delivers must-have education on themes that relate to the overall business transformation taking place in the communications industry.
    LIVE NOW!
    Friday, December 2, 1:00PM EST
    The SDN Approach to IP & Optical Integration
    Sterling Perrin, Senior Analyst, Heavy Reading
    UPCOMING COURSE SCHEDULE
    Friday, December 2, 1:00PM EST
    The SDN Approach to IP & Optical Integration
    Sterling Perrin, Senior Analyst, Heavy Reading
    in association with:
    From The Founder
    Light Reading today starts a new voyage as part of a larger Enterprise.
    Flash Poll
    Live Streaming Video
    Charting the CSP's Future
    Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it's going from two industry veterans.
    Women in Comms Introduction Videos
    Korn Ferry Consultant: How to Find, Cultivate & Be the Best Talent

    11|30|16   |   4:10   |   (1) comment


    Erin Callaghan, a managing consultant for Korn Ferry Futurestep, shares strategies for companies to improve how they recruit and for women to ensure they don't get lost in the pipeline.
    LRTV Custom TV
    We Can Make the World More Sustainable

    11|29|16   |     |   (0) comments


    GeSI is a global e-Sustainability Initiative organization bringing together 40 big multinational companies around the world. According to GeSI's report, information and communication technology can make the world more sustainable. Luis Neves, chairman of GeSI, shared with us his opinion at Ultra-broadband Forum (UBBF2016).
    LRTV Custom TV
    Finding a New Way to Engage Customers & Drive Revenue

    11|29|16   |     |   (0) comments


    Mobile revenues are declining. Digicel, a player in the Caribbean telecommunications/entertainment space, has found a new way to engage customers and drive revenue. John Quinn, CTO of Digicel, shared with us its story at Ultra-broadband Forum (UBBF2016)
    LRTV Custom TV
    Do You Really Need Gigabit Infrastructure?

    11|29|16   |     |   (0) comments


    Altibox is the biggest fiber-to-the-home (FTTH) player and the largest provider of video and TV in Norway. They started out with zero customers in 2002. Now they have close to half a million households and companies attached to their FTTH business. Nils Arne, CEO of Altibox shared with us their story and insight on 5G at Ultra-broadband Forum (UBBF2016).
    LRTV Custom TV
    BTís Openreach Strategy & Its Updates in 2016

    11|29|16   |     |   (0) comments


    A lot of developments at Openreach this year in terms of strategy and planned investments. Peter Bell, CIO of Openreach BT, shared with us the updates of Openreach at Ultra-broadband Forum (UBBF2016).
    LRTV Custom TV
    ITU: The Broadband Is Our Future

    11|29|16   |     |   (0) comments


    At Ultra-broadband Forum, Houlin Zhao, Secretary General of ITU, discussed how important it is for countries, companies and everybody to be working together to help to build the broadband and digital economies (UBBF2016).
    LRTV Custom TV
    Tackling 5G in Dallas

    11|28|16   |     |   (0) comments


    Here are our highlights of the 5G North America show in Dallas, Texas with Light Reading's Dan Jones.
    LRTV Interviews
    Cox Prepping for Virtualization Trials

    11|14|16   |     |   (0) comments


    In this video interview, Cox's Jeff Finkelstein discusses MSO's plans to test managed business services in early 2017 and tackle Distributed Access Architectures.
    LRTV Custom TV
    Drivers & Potential of NGP

    11|11|16   |     |   (0) comments


    ETSI has created an Industry Specification Group to work on Next Generation Protocols (NGP ISG), looking at evolving communications and networking protocols to provide the scale, security, mobility and ease of deployment required for the connected society of the 21st century. The NGP ISG will identify the requirements for next generation protocols and network ...
    LRTV Custom TV
    Huawei IP 2020 for Future Networks

    11|11|16   |     |   (0) comments


    Future Networks should satisfy many requirements such as high throughput, extremely low latency, flexible mobility, intrinsic security, networking automation, and so forth. The Chief Architect of Huawei Future Networks addresses a holistic solution, i.e., IP 2020, to achieve these requirements for various future life scenarios (e.g., autonomous driving, tactile ...
    LRTV Custom TV
    Digital Object Architecture

    11|11|16   |     |   (0) comments


    Digital Object Architecture provides a basic information infrastructure that can facilitate interoperability between or among different systems, processes, and other information resources, including different identity management systems. Digital objects are networked objects that are named by digital object identifiers and instantiated by an infrastructure service ...
    LRTV Custom TV
    BT's Openreach Has High Hopes for Long-Reach VDSL

    11|11|16   |   06:04   |   (0) comments


    Peter Bell, Network Portfolio CIO at BT's access business Openreach, talks about the operator's trial of a new broadband access technology called Long Reach VDSL.
    Upcoming Live Events
    December 6-8, 2016, The Westin Excelsior, Rome
    May 16-17, 2017, Austin Convention Center, Austin, TX
    All Upcoming Live Events
    Infographics
    Hot Topics
    AT&T Debuts DirecTV Now on New Video Platform
    Mari Silbey, Senior Editor, Cable/Video, 11/28/2016
    Apple Seeds 5G? Seeks 'Multi-Gigabit' Chip Designer
    Dan Jones, Mobile Editor, 11/30/2016
    Altice Plans FTTH for Entire US Footprint
    Iain Morris, News Editor, 11/30/2016
    Altice FTTH Bill Could Hit Almost $9.6B in US
    Iain Morris, News Editor, 12/1/2016
    Samsung Bows to Investors, Considers Revamp
    Iain Morris, News Editor, 11/29/2016
    Like Us on Facebook
    Twitter Feed
    BETWEEN THE CEOs - Executive Interviews
    Eyal Waldman, CEO of Mellanox Technologies, speaks to Steve Saunders, CEO of Light Reading, for an exclusive interview about the 100 GB cable challenge, cybersecurity and much more.
    Join us for an in-depth interview between Steve Saunders of Light Reading and Alexis Black Bjorlin of Intel as they discuss the release of the company's Silicon Photonics platform, its performance, long-term prospects, customer expectations and much more.
    Live Digital Audio

    Even when there's a strong pipeline of female talent in the comms industry, it tends to leak all the way to the top. McKinsey & Company says women experience pipeline leakage at three primary points: being unable to enter, being stuck in the middle or being locked out of the top. Each pipeline pain point presents its own challenges, but also opportunities to stop the leak. Wireless operator Sprint is making a conscious effort to improve its own pipeline from new recruits to the C-suite, and it wants the rest of the industry to do the same. In this Women in Comms radio show, WiC Board Member and Sprint Vice President of Enterprise Sales Nelly Pitocco will give us her take on the industry's pipeline challenges. Pitocco, who joined Sprint in May and has spent 20 years in the comms industry, will also offer solutions, share how Sprint is tackling the challenge within its own organization and take your questions live on air.