Light Reading

The 400Gbit/s Demonstrator

Eduard Beier
8/21/2013
100%
0%

A collaboration between research institutions and industrial partners demonstrated at the ISC'13 that 400Gbit/s bandwidth granularity is not only feasible, but already useful today.

For demonstration purposes a number of innovative technologies such as 400Gbit/s DWDM Super Channel, high-speed flash memory and a distributed parallel file system are used in combination.

The project is set up to have real data and realistic applications instead of yet another "hero experiment" with test generators and analyzers. Initial performance tests were performed to check the operational condition of all components working together in the demonstrator. Synthetic data and load was put on the connectivity and IT components and showed overall good operational conditions.

Then two applications were activated on the demonstrator:

  • Climate Research with centralized computing accessing to distributed data

  • Turbine Development with multi-stage processing dynamically both shifting big-data between Munich and Dresden (620km standard fiber) over a single 400Gbit/s wavelength Super Channel.

    From the research community are involved: the German Centre for Aerospace (DLR); the Leibniz Supercomputing Centre (LRZ); the Max Planck Institute for Meteorology (MPI-M) from the Max Planck Society and the computing center of the Max Planck Society in Garching (RZG); the computer center of the Technical University of Freiberg (RZ TUBAF); and the Center for Information Services and High Performance Computing Dresden University of Technology (ZIH).

    Industrial partners are: Alcatel-Lucent (NYSE: ALU), Barracuda Networks Inc. , Bull SA , Cluster Vision, EMC2, IBM Corp. (NYSE: IBM), Mellanox Technologies Ltd. (Nasdaq: MLNX), Deutsche Telekom AG (NYSE: DT), and T-Systems International GmbH .

    The fundamental network structure of the Demonstrator is shown by the following picture:

    Network Structure

    The compute clusters on both sides are clients of the distributed parallel file system which consists of 12 servers with three flash memory boards in each server. Both clusters are able to read and write at 400 Gbit/s on that file system.

    Standard Sandy Bridge servers with at least five PCIe3.0x8 slots and 128GB DRAM are used. The design goal was to create about 18 GBytes of sustained duplex data rate per node (Ethernet: 5GBytes, Memory: 6 GBytes, Fabric: 7 GBytes). The Fabric data rate has not yet been tested, the other rates have been confirmed by tests.

    For performance margin reasons the setup is moderately overbooked between memory and network. Below are the total theoretical performance numbers.

    Theoretical Throughput

    The predecessor project in 2010 (100Gbit/s Testbed Dresden Freiberg) required arrays of about 800 spinning disks on each side to form a 100Gbit/s data path. Because that setup was, for scaling reasons, not possible for 400 Gbit/s, the 2 GBytes I/O throughput flash memory board is one of the enablers of this 400Gbit/s project.

    For the ISC configuration (in the picture below), the following systems were added: a 10GbE Cluster in Dresden; a 2x100GbE link to Freiberg (not connected during ISC); in Garching the SuperMUC at LRZ (one of the fastest supercomputers in Europe) and a 480 GBytes cluster (> 2500 cores) at RZG; and a commercial Cloud Service from T-Systems (not connected during ISC).

    The Big Picture

    The project links to SDN and NFV -- in particular as it is very active in NSI (Network-to-Service-Interface) definition. (For further information, see the link below to additional project information).

    Because the "distributed high speed GPFS" approach is of some universal nature (e.g. for HPC datacenter backup and HPC workload distribution reasons) the setup will be tested for commercial applicability during the post ISC phase. The possibility to use network functions like encryption, firewalling and data compression is definitely a must in a commercial case.

    Network appliances for 40 Gbit/s and 100 Gbit/s are neither available nor would they be affordable in many cases. Therefore we are going to test virtualized network functions on standard server hardware (see picture below); that additional "module," which is based on the same server hardware as the other servers, gets in between each server and the router on each side.

    NFV Module

    Please feel free to ask questions about any technical aspect of the project. The project team, which consists of some of the best specialists on their individual field, will be happy to answer. For additional project information: http://tu-dresden.de/zih/400gigabit.

    The Partners

    — Eduard Beier, process systems engineer, T-Systems International

    (6)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View
    RolfSperber
    50%
    50%
    RolfSperber,
    User Rank: Light Beer
    9/26/2013 | 10:11:00 AM
    Re: SDN and NFV
    Ray, we are at a very early stage, but abstracting from both hardware (and in consequence IOS) layer and utilizing a common framework (see NSI WG in OGF) and at the same time allow for docking of virtalized network functions will work for multi vendor, multi carrier and multi domain. Still, its a long way to go!
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    9/26/2013 | 7:51:13 AM
    Re: SDN and NFV
    Rolf

    You say it is not restricted to a single domain, but is it applicable in networks that traverse multiple infrastructures run and managed by multiple network operators?
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:43:49 AM
    Re: Is this another route in for NFV?
    Requirements in industry will not be so different from those in R&D. Looking at the plans in the context of Horizon 2020 Puplic Private Partnership is a target of European efforts. Taking into account the cost of infrastructure and the prevailing attitude to pay as little as possible for utilization of infrastructure, more sophisticated multiplexing methods, and this include NFV, are inevitable.

    For network operators this scheme means significantly reduced time to market.
    RolfSperber
    100%
    0%
    RolfSperber,
    User Rank: Light Beer
    9/25/2013 | 8:36:55 AM
    SDN and NFV
    In this project we will be going a step further. Our plan is to create an environment that allows for creating a virtual network based on the requirements of either applications or  carrier provided network functionality. We will not be restricted to a single domain and connectivity, our target is a network created from building blocks out of a repository. These building blocks can be connectivity with certain quality parameters or virtualized network functions like e.g. firwall functionality, compression, encryption, accelleration.

     

     
    Eddie_
    50%
    50%
    Eddie_,
    User Rank: Blogger
    8/21/2013 | 3:00:47 PM
    Re: Is this another route in for NFV?
    NFV simply scales better than HW based approaches (if NFV tests in September show good results).

    A possible 2013 roadmap for the project:
    • fully SDN controlled network
    • 200GBit/s datapath
    • NFV in the data path 

    a possible 2014 roadmap for the project:
    • scale up to a 1TBit/s data path

    how else could you do that?

     

     

     

     

     
    Ray@LR
    50%
    50%
    Ray@LR,
    User Rank: Blogger
    8/21/2013 | 1:01:52 PM
    Is this another route in for NFV?
    Interesting that functions virtualization takes the place of appliances that are either too expensive to deploy or have not yet been created... is NFV going to help the R&D sector more than production network operatins in the early years?
    Educational Resources
    sponsor supplied content
    Educational Resources Archive
    More Blogs from Column
    Wi-Fi Alliance members have created Wi-Fi Aware technology to bring local experiences to life -- without requiring access to the Internet or GPS.
    Bamboozled by the flood of abbreviations and acronyms that SDN and NFV have inspired? Here's a guide to some of the key standards bodies whose initials you will need to know.
    Networks of the future will rely on "white box" switches and servers rather than proprietary hardware and that's going to alter the shape of the communications industry. Who says so? John Chambers.
    There are a number of key questions to consider before any SDN migration plans are put into action.
    Bigger. And Better. But definitely bigger.
    Flash Poll
    From The Founder
    Networks of the future will rely on "white box" switches and servers rather than proprietary hardware and that's going to alter the shape of the communications industry. Who says so? John Chambers.
    LRTV Huawei Video Resource Center
    Huawei's Power Play With GrupoGiro

    4|1|15   |   2:51   |   (0) comments


    Daniel Heredia, CEO of Energiro, the power management services part of Spain's Grupo Giro, explains why his company has just struck a partnership with Huawei for uninterruptible power supply products and how he hopes it can take the partnership into other markets.
    LRTV Huawei Video Resource Center
    Huawei Intros Smart Device for eLTE

    3|30|15   |   05:25   |   (0) comments


    Huawei has developed a secure, location-aware multimedia smartphone for its eLTE trunked radio solution, says Huawei's Norman Frisch.
    LRTV Huawei Video Resource Center
    Win Video, Win All

    3|30|15   |   06:44   |   (0) comments


    Video is going to be the next main source of revenue for operators. Operators have big opportunities and advantages to monetize video services. Globally, Huawei has helped more than 70 operators achieve over 30 million video subscribers. Watch this video for more.
    LRTV Custom TV
    The Benefits of HyperScale Clouds for NFV

    3|27|15   |   01:50   |   (0) comments


    Hyperscale cloud has been developed by the Internet giants to support the creation and delivery of software-based services at blistering speeds, and at the lowest possible cost. The original ETSI NFV vision was to adopt hyperscale cloud architecture and practices. This vision has become somewhat obscured along the way, due to misunderstandings about the hyperscale ...
    LRTV Huawei Video Resource Center
    eLTE Rapid Meets the Need for Speed

    3|26|15   |   4:45   |   (0) comments


    Designed especially for emergency and dedicated ad hoc local mobile communications coverage, Huawei's eLTE Rapid solution can deliver trunked voice, video and data coverage for multiple users over a 6km range and be set up in just 15 minutes, explains Huawei's Norman Frisch.
    LRTV Huawei Video Resource Center
    On Videos: Challenges & Opportunities

    3|26|15   |   5:56   |   (0) comments


    Most everything is now connected. And along with 4K and 4G technologies, everyone could be creating and broadcasting video contents. Users are expecting better video experience with any screen, anywhere and anytime. Operators will meet new challenges, but also see some big opportunities.
    LRTV Custom TV
    JDSU: Delivering Dynamic Networks for a Personalized Experience

    3|26|15   |   5:59   |   (0) comments


    Light Reading speaks to JDSU at Mobile World Congress 2015 about new solutions in the areas of HetNets, VoLTE, backhaul, virtualization, big data analytics, and real-time intelligence.
    LRTV Custom TV
    Smarter Service Chaining & New Ways to Benefit From Qosmos Technology

    3|25|15   |   03:11   |   (0) comments


    David Le Goff, director of strategic and product marketing at Qosmos, explains how the company has added application awareness to subscriber information to make service chaining more efficient and reduce costs for networking and infrastructure. In addition, Qosmos technology, which has been delivered as C libraries, is now also available as a virtual machine, ...
    Between the CEOs
    Qosmos CEO: The Changing Face of DPI

    3|24|15   |   13:53   |   (0) comments


    LR CEO and Founder Steve Saunders sits down with the head of Qosmos to talk about the changing state of the art in deep packet inspection technology, including its role in SDN and NFV architectures. Also, how the comms market is becoming more like the automotive industry.
    LRTV Huawei Video Resource Center
    FC Schalke Scores With Its Agile Stadium

    3|24|15   |   6:23   |   (0) comments


    Top German soccer club FC Schalke 04 has deployed a new, agile WiFi network from Huawei in its Veltins-Arena stadium and is reaping the benefits in terms of customer satisfaction and business opportunities, explains marketing chief Alexander Jobst.
    LRTV Huawei Video Resource Center
    Huawei’s Insights on Mobile Video

    3|24|15   |   7:51   |   (0) comments


    More people than ever are now watching videos on smartphones. Seventy percent of mobile traffic will be video traffic until 2018. In this video, Huawei's exports give their insights on mobile video in terms of business model, network planning and 4G network construction.
    LRTV Documentaries
    The Rise of Industry 4.0

    3|24|15   |   02:26   |   (9) comments


    Are you ready for the fourth industrial revolution? It's a big deal for influential operators such as Deutsche Telekom.
    Upcoming Live Events
    April 14, 2015, The Westin Times Square, New York City, NY
    May 5, 2015, Hyatt McCormick Place, Chicago, IL
    May 6, 2015, Georgia World Congress, Atlanta, GA
    May 12, 2015, Grand Hyatt, Denver, CO
    May 13-14, 2015, The Westin Peachtree, Atlanta, GA
    June 8, 2015, Chicago, IL
    June 9-10, 2015, Chicago, IL
    June 9, 2015, Chicago, IL
    June 10, 2015, Chicago, IL
    September 29-30, 2015, The Westin Grand Müchen, Munich, Germany
    November 11-12, 2015, The Westin Peachtree Plaza, Atlanta, GA
    All Upcoming Live Events
    Infographics
    More and more traffic is traveling within data centers, and operators need new networking approaches to manage this growth. Pica8's infographic shows how operators can confidently and painlessly deploy white box switches into existing networks today and leverage the benefits of SDN.
    Hot Topics
    AT&T Woos SMBs With Small-Scale WiFi
    Sarah Thomas, Editorial Operations Director, 3/26/2015
    Just Don't Say IBM Is 'Relaunching' Networking Business
    Mitch Wagner, West Coast Bureau Chief, Light Reading, 3/26/2015
    TV Everywhere Nears Mainstream Adoption
    Mari Silbey, Independent Technology Editor, 3/27/2015
    Carriers Are Bright Spot in BlackBerry Q4
    Sarah Thomas, Editorial Operations Director, 3/27/2015
    Comcast Says TWC Deal Will Close Later
    Mari Silbey, Independent Technology Editor, 3/26/2015
    Like Us on Facebook
    Twitter Feed
    Webinar Archive
    BETWEEN THE CEOs - Executive Interviews
    LR CEO and Founder Steve Saunders sits down with the head of Qosmos to talk about the changing state of the art in deep packet inspection technology, including its role in SDN and NFV architectures.
    Chattanooga’s EPB publicly owned utility comms company has become a poster child for how to enable a local economy using next-gen networking technology. Steve Saunders, Founder of Light Reading, sits down with Harold DePriest, president and CEO of EPB, to learn how EPB is bringing big time tech to small town America.