& cplSiteName &

EXCLUSIVE! NFV Interop Evaluation Results

Light Reading
Prime Reading
Light Reading
12/8/2015
100%
0%

Of all the industry test results out there, this is the one that people have been waiting for.

In a historic first for the communications networking industry, Light Reading, in partnership with EANTC on behalf of the New IP Agency (NIA), is happy and proud to announce the publication of the world's first independent interoperability evaluation of NFV infrastructure, focused (in Phase 1) on multivendor NFV infrastructure-to-virtual network function (VNF) interoperability.

These results are just a first step on a very important journey for an industry that is in dramatic upheaval. Never before have network operators, faced with the unknown and daunting challenges of introducing virtualized functions and their supporting infrastructure platforms, needed independent test results as much as they need them now: Such independent evaluations can cut down development times, save operators resources, and help speed up further tests, field trials and live deployments.

That's why we have been moving fast. The Phase 1 evaluation, which was publicized, set up and completed in just a few months (an extraordinarily short time for such a procedure), included 12 vendors, which between them submitted four NFV infrastructure (NFVi) platforms and 17 VNFs.

The NFVi vendors, which all offered platforms based on OpenStack, were:

  • Alcatel-Lucent
  • Cisco Systems
  • Huawei Technologies
  • Juniper Networks

The VNF vendors were:

  • Alcatel-Lucent
  • Cisco Systems
  • Cobham Wireless
  • Hitachi Communication Technologies America
  • Huawei Technologies
  • IneoQuest Technologies
  • Juniper Networks
  • Metaswitch Networks
  • NetNumber
  • Netrounds
  • Procera Networks
  • Sonus Networks

The VNFs came in many shapes and forms, from virtual routers, EPC (evolved packet core) and IMS components, firewalls, test probes, deep packet inspection (DPI) modules and session border controllers.

The testing focused on the functional interoperability of NFVi platforms and various VNFs, but did not include the sort of tests more commonly associated with communications infrastructure, such as performance, scalability and resilience testing: In addition, these tests did not include any management and orchestration (MANO) evaluations (they will be the focus of a later evaluation phase).

The tests carried out by the EANTC team, which were all conducted remotely (another new feature!), were of NFVi-VNF interoperability combinations that had never been tried before in such open conditions. As a result, not all were successful, but as this is still a very nascent market, it was expected that some of the evaluations were not be completed successfully.

In fact, the success rate -- 25 of the 39 combinations tested, or 64%, passed -- was "a great result," noted EANTC managing director Carsten Rossenhövel.

That higher-than-expected pass rate was due, in part, to the high degree of industry participation: During the seven weeks of testing, 55 engineers from various vendors (including test system partner Ixia) were involved, with support from 14 marketing and communications executives, seven EANTC staff and three from Light Reading/NIA: In total, 79 people were directly involved in the process.

So does that mean that there was a 36% fail rate? It could be viewed that way, but that would be shortsighted. The point of these evaluations is not to identify technology that 'does not work' – it is to see what can be done better, what needs to be fixed. The NIA exists to advance the industry and speed up the process of getting useful technology to market: When the evaluations highlight an interoperability combination that doesn't work, that means the vendors can fix the problems! This can only be good for the industry.

The evaluation also highlighted many other interesting and useful outcomes, not least of which was insight into the performance of OpenStack. For some OpenStack supporters, it is a viable alternative to standardization: "If only everybody would base their implementations on OpenStack, all interop issues would be solved," could be a phrase you might hear in virtualization circles.

Well, according to EANTC's results, OpenStack is far from ready to play an interop role. "There were tons of interop issues despite the fact that all NFVis were based on OpenStack," noted EANTC's Rossenhövel. That's because there are far too many options (including how to manage a VNF), too many versions (with backwards compatibility an oversight), and items left open for vendor consideration (including license management).

So there are many positives to take away from the Phase 1 test results, but there is much, much more to be done: A test combination that passed confirms that the initial lifecycle management tasks work between the parties on a functional basis, but that still leaves many aspects of application-layer functionality, performance, resiliency and orchestration to be evaluated. Those aspects will be the focus of future evaluations.

In the meantime, the following pages will provide you will a full matrix of the test results and detailed insights into the evaluations, the outcomes and the conclusions. So dive in!

Page 2: The Participants

Page 3: Test Setup and Coverage

Page 4: The Results Matrix

Page 5: Interoperability Findings and Challenges

Page 6: Further Interoperability Findings and Challenges

  • Prefer to read this report in a downloadable PDF file? An extended PDF version of this report, which includes individual report cards for each test combination that achieved a 'pass,' is available for free (to any Light Reading registered user) at this link.

    — Ray Le Maistre, Circle me on Google+ Follow me on TwitterVisit my LinkedIn profile, Editor-in-Chief, Light Reading and Carsten Rossenhövel, Managing Director, European Advanced Networking Test Center AG (EANTC) (http://www.eantc.de/), an independent test lab in Berlin. EANTC offers vendor-neutral network test facilities for manufacturers, service providers, and enterprises.

    Next page: The Participants

    (2)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    muzza2011
    50%
    50%
    muzza2011,
    User Rank: Light Beer
    12/10/2015 | 2:39:46 PM
    Network now needs a permanent Proof of Concept lab
    We're at the 'art of the possible' stage, rather than the 'start of the probable' regarding live deployment... which if taken at face value, should wholly sidestep the heinous blunder of productionising a nine 5's infrastructure in for a five 9's expectation.

    As much as the IT proposition to a user involves all layers of the ISO stack, if you screw the network you've lost the farm, which then negates any major investment in state of the art delivery systems up the stack.

    Any tech worth their weight will crash and burn this stuff on Proof of Concept lab... and weld all the fork doors closed... as it'll be their 'nads on the line should if fail in production, not any IT upper heirarchy who decided to make an (unwise) executive decision.

    Fail means *anything* regardless whether its a SNAFU or an hack-attack vector that no-one tested. 

    In short, test to destruction, armour-plate the resultant design, and always have a PoC lab bubbling away in the background for the next iteration, because the genie is now out of the bottle and won't go back in.
    mhhf1ve
    100%
    0%
    mhhf1ve,
    User Rank: Light Sabre
    12/8/2015 | 8:56:52 PM
    Open source doesn't necessarily mean any support available...
    It's always interesting to see how open source platforms still have gaping holes with major unsupported functions. Every fork/flavor of an open source project means more than a few potential gaps for interoperability with other forks.

     
    From The Founder
    The independent evaluation of Nokia's key virtual network functions (VNFs) was a defining moment for the Finnish giant.
    Flash Poll
    Live Streaming Video
    Charting the CSP's Future
    Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it's going from two industry veterans.
    LRTV Custom TV
    Energy 2020: Growing Services, Not Consumption

    6|24|16   |   07:18   |   (0) comments


    Management of power requirements needs to be a key consideration as cable operators deploy new services, says Dan Cooper, vice president of critical infrastructure for Charter Communications and chair of the SCTE/ISBE Standards Program's Energy Management Subcommittee. In this video, Cooper discusses the importance of cable operators and technology partners ...
    LRTV Huawei Video Resource Center
    Huawei & TDC First to Launch DOCSIS 3.1 Trial in Europe

    6|24|16   |     |   (0) comments


    To meet the rising demands for high-speed broadband and to tackle fierce competition for 1.5 million household, TDC wanted to upgrade its network to use the latest technology. Using Huawei's DOCSIS 3.1 solution, TDC is now able to offer gigabit speeds to its customers.
    LRTV Documentaries
    OPNFV Summit: Key Takeaways

    6|22|16   |   03:28   |   (0) comments


    MANO shortfalls, an increasingly bloated open source ecosystem and the cultural challenges for network operators were among the key takeaways from the OPNFV Summit in Berlin.
    LRTV Huawei Video Resource Center
    Bringing Ultra Broadband to the South Pacific

    6|22|16   |   7:55   |   (0) comments


    At Singapore's UBBS 2016 summit, Light Reading speaks to Lim Chee Siong, Huawei's CMO in the South Pacific Region.
    LRTV Custom TV
    Juniper's NFV Platform

    6|22|16   |     |   (0) comments


    A telco cloud is a key enabler for service providers to deliver virtualized services to their customers. Juniper has a differentiated approach with an open and comprehensive NFV platform to build the cloud of the future for multiple use cases.
    Women in Comms Introduction Videos
    Infinera: The Glass Ceiling's Been Broken

    6|22|16   |   4:34   |   (0) comments


    Shannon Williams, an engineer by trade and now the director of sales for Infinera's major accounts, draws confidence from being a female in a male-dominated industry and hopes other women will too as industry dynamics swing in our favor.
    LRTV Custom TV
    RAD Demonstrates SD-WAN Functionality at BCE 2016

    6|22|16   |     |   (0) comments


    In conjunction with Versa Networks, RAD demos a joint solution for SD-WAN implementation. The solution uses RAD's ETX-2i, a vCPE platform integrating a powerful IP and Carrier Ethernet NID/NTU with a field-pluggable x86 NFV module that runs Versa's FlexVNF for SD-WAN functionality. The demo implements SD-WAN over a simulated service provider footprint by deploying ...
    LRTV Custom TV
    Radware on Virtualizing Network Security

    6|21|16   |     |   (0) comments


    Radware's Michael O'Malley highlights the benefits for service providers of virtualizing network security.
    LRTV Huawei Video Resource Center
    Huawei 2016 UBBS World Tour: Singapore

    6|21|16   |   02:39   |   (0) comments


    Highlights from Huawei's UBBS World Tour event in Singapore.
    LRTV Custom TV
    The FlowBROKER Solution

    6|21|16   |     |   (0) comments


    During a demo at the Big Communications Event 2016, Accedian director of product line solutions Greg Spear demonstrates how the FlowBROKER distributed packet capture solution (part of Accedian's SkyLIGHT platform) can be used to troubleshoot and resolve QoE issues.

    FlowBROKER is the industry's first lossless remote packet capture solution, which uses ...

    LRTV Documentaries
    BCE 2016: Dell & the New World Order

    6|21|16   |   22:48   |   (0) comments


    Dell's Jim Ganthier on how the very essence of enterprise is being affected by digital disruption – and how the cloud can help companies find a path forward.
    LRTV Documentaries
    BCE 2016: The CSP of the Future

    6|20|16   |     |   (0) comments


    Traditional telecom operators have been adopting new ways of doing business from the IT sector and emulating web-scale companies in their technology adoption. Are these strategies succeeding? A panel of experts at BCE considered the evidence...
    Upcoming Live Events
    September 13-14, 2016, The Curtis Hotel, Denver, CO
    November 3, 2016, The Montcalm Marble Arch, London
    November 30, 2016, The Westin Times Square, New York City
    December 6-8, 2016,
    May 16-17, 2017, Austin Convention Center, Austin, TX
    All Upcoming Live Events
    Infographics
    A new survey conducted by Heavy Reading and TM Forum shows that CSPs around the world see the move to digital operations as a necessary part of their overall virtualization strategies.
    Hot Topics
    FCC to Vote on 5G Spectrum on July 14
    Dan Jones, Mobile Editor, 6/20/2016
    'Brexit' Vote Hits BT, Vodafone
    Iain Morris, News Editor, 6/24/2016
    AT&T Settles on LTE for Cellular IoT
    Dan Jones, Mobile Editor, 6/22/2016
    Google Fiber Buys Webpass in Wireless Play
    Iain Morris, News Editor, 6/23/2016
    Arista Could Face US Import Ban – Reports
    Mitch Wagner, West Coast Bureau Chief, Light Reading, 6/23/2016
    Like Us on Facebook
    Twitter Feed
    BETWEEN THE CEOs - Executive Interviews
    At the BCE 2016 show in Austin, ECI Telecom CEO Darryl Edwards tells Light Reading founder and CEO about the Elastic Network concept and the company's NFV and cybersecurity developments.
    Mark Dzuban, president and CEO of SCTE/ISBE, sits down with Steve Saunders in Light Reading's New York office to discuss the society's Energy 2020 campaign, including its mission to benefit the environment, enable economic benefits and the key challenges facing cable operators wanting to reduce energy consumption.
    Animals with Phones
    Live Digital Audio

    Our world has evolved through innovation from the Industrial Revolution of the 1740s to the information age, and it is now entering the Fourth Industrial Revolution, driven by technology. Technology is driving a paradigm shift in the way digital solutions deliver a connected world, changing the way we live, communicate and provide solutions. It can have a powerful impact on how we tackle some of the world’s most pressing problems. In this radio show, Caroline Dowling, President of Communications Infrastructure & Enterprise Computing at Flex, will join Women in Comms Director Sarah Thomas to discuss the impact technology has on society and how it can be a game-changer across the globe; improving lives and creating a smarter world. Dowling, a Cork, Ireland, native and graduate of Harvard Business School's Advanced Management Program, will also discuss her experience managing an international team focused on innovation in an age of high-speed change.