& cplSiteName &

EXCLUSIVE! NFV Interop Evaluation Results

Light Reading
Prime Reading
Light Reading
12/8/2015
100%
0%

Of all the industry test results out there, this is the one that people have been waiting for.

In a historic first for the communications networking industry, Light Reading, in partnership with EANTC on behalf of the New IP Agency (NIA), is happy and proud to announce the publication of the world's first independent interoperability evaluation of NFV infrastructure, focused (in Phase 1) on multivendor NFV infrastructure-to-virtual network function (VNF) interoperability.

These results are just a first step on a very important journey for an industry that is in dramatic upheaval. Never before have network operators, faced with the unknown and daunting challenges of introducing virtualized functions and their supporting infrastructure platforms, needed independent test results as much as they need them now: Such independent evaluations can cut down development times, save operators resources, and help speed up further tests, field trials and live deployments.

That's why we have been moving fast. The Phase 1 evaluation, which was publicized, set up and completed in just a few months (an extraordinarily short time for such a procedure), included 12 vendors, which between them submitted four NFV infrastructure (NFVi) platforms and 17 VNFs.

The NFVi vendors, which all offered platforms based on OpenStack, were:

  • Alcatel-Lucent
  • Cisco Systems
  • Huawei Technologies
  • Juniper Networks

The VNF vendors were:

  • Alcatel-Lucent
  • Cisco Systems
  • Cobham Wireless
  • Hitachi Communication Technologies America
  • Huawei Technologies
  • IneoQuest Technologies
  • Juniper Networks
  • Metaswitch Networks
  • NetNumber
  • Netrounds
  • Procera Networks
  • Sonus Networks

The VNFs came in many shapes and forms, from virtual routers, EPC (evolved packet core) and IMS components, firewalls, test probes, deep packet inspection (DPI) modules and session border controllers.

The testing focused on the functional interoperability of NFVi platforms and various VNFs, but did not include the sort of tests more commonly associated with communications infrastructure, such as performance, scalability and resilience testing: In addition, these tests did not include any management and orchestration (MANO) evaluations (they will be the focus of a later evaluation phase).

The tests carried out by the EANTC team, which were all conducted remotely (another new feature!), were of NFVi-VNF interoperability combinations that had never been tried before in such open conditions. As a result, not all were successful, but as this is still a very nascent market, it was expected that some of the evaluations were not be completed successfully.

In fact, the success rate -- 25 of the 39 combinations tested, or 64%, passed -- was "a great result," noted EANTC managing director Carsten Rossenhövel.

That higher-than-expected pass rate was due, in part, to the high degree of industry participation: During the seven weeks of testing, 55 engineers from various vendors (including test system partner Ixia) were involved, with support from 14 marketing and communications executives, seven EANTC staff and three from Light Reading/NIA: In total, 79 people were directly involved in the process.

So does that mean that there was a 36% fail rate? It could be viewed that way, but that would be shortsighted. The point of these evaluations is not to identify technology that 'does not work' – it is to see what can be done better, what needs to be fixed. The NIA exists to advance the industry and speed up the process of getting useful technology to market: When the evaluations highlight an interoperability combination that doesn't work, that means the vendors can fix the problems! This can only be good for the industry.

The evaluation also highlighted many other interesting and useful outcomes, not least of which was insight into the performance of OpenStack. For some OpenStack supporters, it is a viable alternative to standardization: "If only everybody would base their implementations on OpenStack, all interop issues would be solved," could be a phrase you might hear in virtualization circles.

Well, according to EANTC's results, OpenStack is far from ready to play an interop role. "There were tons of interop issues despite the fact that all NFVis were based on OpenStack," noted EANTC's Rossenhövel. That's because there are far too many options (including how to manage a VNF), too many versions (with backwards compatibility an oversight), and items left open for vendor consideration (including license management).

So there are many positives to take away from the Phase 1 test results, but there is much, much more to be done: A test combination that passed confirms that the initial lifecycle management tasks work between the parties on a functional basis, but that still leaves many aspects of application-layer functionality, performance, resiliency and orchestration to be evaluated. Those aspects will be the focus of future evaluations.

In the meantime, the following pages will provide you will a full matrix of the test results and detailed insights into the evaluations, the outcomes and the conclusions. So dive in!

Page 2: The Participants

Page 3: Test Setup and Coverage

Page 4: The Results Matrix

Page 5: Interoperability Findings and Challenges

Page 6: Further Interoperability Findings and Challenges

  • Prefer to read this report in a downloadable PDF file? An extended PDF version of this report, which includes individual report cards for each test combination that achieved a 'pass,' is available for free (to any Light Reading registered user) at this link.

    — Ray Le Maistre, Circle me on Google+ Follow me on TwitterVisit my LinkedIn profile, Editor-in-Chief, Light Reading and Carsten Rossenhövel, Managing Director, European Advanced Networking Test Center AG (EANTC) (http://www.eantc.de/), an independent test lab in Berlin. EANTC offers vendor-neutral network test facilities for manufacturers, service providers, and enterprises.

    Next page: The Participants

    (2)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    muzza2011
    50%
    50%
    muzza2011,
    User Rank: Light Beer
    12/10/2015 | 2:39:46 PM
    Network now needs a permanent Proof of Concept lab
    We're at the 'art of the possible' stage, rather than the 'start of the probable' regarding live deployment... which if taken at face value, should wholly sidestep the heinous blunder of productionising a nine 5's infrastructure in for a five 9's expectation.

    As much as the IT proposition to a user involves all layers of the ISO stack, if you screw the network you've lost the farm, which then negates any major investment in state of the art delivery systems up the stack.

    Any tech worth their weight will crash and burn this stuff on Proof of Concept lab... and weld all the fork doors closed... as it'll be their 'nads on the line should if fail in production, not any IT upper heirarchy who decided to make an (unwise) executive decision.

    Fail means *anything* regardless whether its a SNAFU or an hack-attack vector that no-one tested. 

    In short, test to destruction, armour-plate the resultant design, and always have a PoC lab bubbling away in the background for the next iteration, because the genie is now out of the bottle and won't go back in.
    mhhf1ve
    100%
    0%
    mhhf1ve,
    User Rank: Light Sabre
    12/8/2015 | 8:56:52 PM
    Open source doesn't necessarily mean any support available...
    It's always interesting to see how open source platforms still have gaping holes with major unsupported functions. Every fork/flavor of an open source project means more than a few potential gaps for interoperability with other forks.

     
    From The Founder
    Cisco's Conrad Clemson, recently promoted to head up the company's Service Provider Apps & Platforms developments, talks to Light Reading's Founder and CEO Steve Saunders about how he's bringing cloud video, mobile and virtualization together to empower network operators.
    Flash Poll
    Live Streaming Video
    Charting the CSP's Future
    Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it's going from two industry veterans.
    LRTV Custom TV
    Cisco's Cloud Scale Networking: Automation, Virtualization & Simplification

    1|18|17   |     |   (0) comments


    Cisco's Sanjeev Mervana outlines the latest innovations in networking technology at CES 2017 in Las Vegas.
    LRTV Custom TV
    ADVA Talks Innovation & the Future of Networking

    1|17|17   |     |   (0) comments


    Ray Le Maistre and Christoph Glingener, CTO of ADVA Optical Networking, discuss the current state of the industry, cooperation and collaboration, open innovation and the future of networking.
    LRTV Custom TV
    Cisco's Infinite Video Platform

    1|17|17   |     |   (0) comments


    Cisco's Infinite Video Platform allows service providers to deliver broadcast-quality video over IP networks. Infinite video supports many devices, from 4K TVs to tablets to game consoles. Join Cisco's Rajeev Raman for a brief tour and live demo.
    LRTV Interviews
    Masergy: Ability to Adapt Key for NFV

    1|16|17   |   6:40   |   (0) comments


    Speaking at Light Reading's 2020 Vision in Rome, Masergy's VP, Global Technology, Ray Watson, said agility is key to providing the mix and match NFV-based services that are driving business for the managed service provider today.
    LRTV Interviews
    Equinix: The Data Explosion

    1|13|17   |   4:16   |   (0) comments


    At Light Reading's 2020 Vision in Rome, Eric Schwartz, president of EMEA, Equinix, talked about how Equinix is helping its customers manage the influx of data today, and how it's preparing for a future filled with millions of connected IoT devices.
    LRTV Interviews
    Heavy Reading: The Changing Data Center Landscape

    1|12|17   |   6:05   |   (1) comment


    At Light Reading's 2020 Vision event in Rome, Heavy Reading's Senior Analyst Roz Roseboro talks about how virtualization is impacting data center evolution and how that evolution is affecting the relationship between service providers, data center operators and public cloud providers.
    LRTV Interviews
    Boingo: Prepping for Millions of Devices

    1|12|17   |   5:07   |   (1) comment


    At Light Reading's 2020 Vision in Rome, Boingo's CTO Derek Peterson discusses how wireless operators will address the needs of low-bandwidth and high-bandwidth apps at the same time, the need for more MHz, the impact of IoT and more.
    LRTV Interviews
    Comcast Shows Off Gig Gateway at CES

    1|11|17   |     |   (1) comment


    With its largest presence at CES in years, Comcast took the wraps off its long-awaited gigabit gateway and a new platform for managing the home WiFi network. Light Reading Senior Editor Mari Silbey sat down with EVP Chris Satchell to discuss the latest Comcast advance, and met with VP of Product Strategy and Development Andrea Peiro to walk through a demo of the ...
    LRTV Interviews
    Colt: End-to-End Key for 2017

    1|10|17   |   6:21   |   (0) comments


    At Light Reading's 2020 Vision Executive Summit in Rome, Nico Fischbach of Colt said having a multi-carrier, end-to-end service proposition is going to be key for 2017 -- and SD-WAN is instrumental in making it happen.
    From the Founder
    Cisco's Clemson on Mobile Cloud Video

    1|9|17   |     |   (1) comment


    Cisco's Conrad Clemson, recently promoted to head up the company's Service Provider Apps & Platforms developments, talks to Light Reading's Founder and CEO Steve Saunders about how he's bringing cloud video, mobile and virtualization together to empower network operators. "If you think about where we're going… whether it's a mobile application, or a video ...
    LRTV Custom TV
    VMware Telco NFV Solutions – Preparing for 5G & IOT

    1|9|17   |     |   (0) comments


    Shekar Ayyar, EVP & Corporate Strategy/General Manager of Telco for VMware, discusses VMware's Telco NFV solutions role and foundation for the Imminent Arrival of 5G & IOT.
    LRTV Interviews
    Heavy Reading: Big Video to Disrupt

    1|6|17   |   4:39   |   (0) comments


    At Light Reading's 2020 Vision Executive Summit in Rome, Heavy Reading's Adi Kishore talks about the challenges of managing and monetizing bandwidth-intensive video, and how service providers will need to transform their networks to cope with the big video explosion.
    Upcoming Live Events
    May 15-17, 2017, Austin Convention Center, Austin, TX
    All Upcoming Live Events
    Infographics
    Hot Topics
    Ericsson: 5G Heralds 'New' New Economy
    Mari Silbey, Senior Editor, Cable/Video, 1/12/2017
    Next Plugfest Gets G.fast Closer to Market
    Brian Santo, Senior editor, Test & Measurement / Components, Light Reading, 1/13/2017
    A Women in Comms Glossary
    Sarah Thomas, Director, Women in Comms, 1/18/2017
    TV's Paradox: No HDR Without 4K
    Mari Silbey, Senior Editor, Cable/Video, 1/17/2017
    Cable Has One Thing Verizon Needs
    Mari Silbey, Senior Editor, Cable/Video, 1/17/2017
    Like Us on Facebook
    Twitter Feed
    BETWEEN THE CEOs - Executive Interviews
    Light Reading founder and CEO Steve Saunders chats with Sportlogiq CEO Craig Buntin about sports data analysis.
    Eyal Waldman, CEO of Mellanox Technologies, speaks to Steve Saunders, CEO of Light Reading, for an exclusive interview about the 100 GB cable challenge, cybersecurity and much more.
    Animals with Phones
    Live Digital Audio

    Playing it safe can only get you so far. Sometimes the biggest bets have the biggest payouts, and that is true in your career as well. For this radio show, Caroline Chan, general manager of the 5G Infrastructure Division of the Network Platform Group at Intel, will share her own personal story of how she successfully took big bets to build a successful career, as well as offer advice on how you can do the same. We’ll cover everything from how to overcome fear and manage risk, how to be prepared for where technology is going in the future and how to structure your career in a way to ensure you keep progressing. Chan, a seasoned telecom veteran and effective risk taker herself, will also leave plenty of time to answer all your questions live on the air.