& cplSiteName &

EXCLUSIVE! NFV Interop Evaluation Results

Light Reading
Prime Reading
Light Reading
12/8/2015
100%
0%

Of all the industry test results out there, this is the one that people have been waiting for.

In a historic first for the communications networking industry, Light Reading, in partnership with EANTC on behalf of the New IP Agency (NIA), is happy and proud to announce the publication of the world's first independent interoperability evaluation of NFV infrastructure, focused (in Phase 1) on multivendor NFV infrastructure-to-virtual network function (VNF) interoperability.

These results are just a first step on a very important journey for an industry that is in dramatic upheaval. Never before have network operators, faced with the unknown and daunting challenges of introducing virtualized functions and their supporting infrastructure platforms, needed independent test results as much as they need them now: Such independent evaluations can cut down development times, save operators resources, and help speed up further tests, field trials and live deployments.

That's why we have been moving fast. The Phase 1 evaluation, which was publicized, set up and completed in just a few months (an extraordinarily short time for such a procedure), included 12 vendors, which between them submitted four NFV infrastructure (NFVi) platforms and 17 VNFs.

The NFVi vendors, which all offered platforms based on OpenStack, were:

  • Alcatel-Lucent
  • Cisco Systems
  • Huawei Technologies
  • Juniper Networks

The VNF vendors were:

  • Alcatel-Lucent
  • Cisco Systems
  • Cobham Wireless
  • Hitachi Communication Technologies America
  • Huawei Technologies
  • IneoQuest Technologies
  • Juniper Networks
  • Metaswitch Networks
  • NetNumber
  • Netrounds
  • Procera Networks
  • Sonus Networks

The VNFs came in many shapes and forms, from virtual routers, EPC (evolved packet core) and IMS components, firewalls, test probes, deep packet inspection (DPI) modules and session border controllers.

The testing focused on the functional interoperability of NFVi platforms and various VNFs, but did not include the sort of tests more commonly associated with communications infrastructure, such as performance, scalability and resilience testing: In addition, these tests did not include any management and orchestration (MANO) evaluations (they will be the focus of a later evaluation phase).

The tests carried out by the EANTC team, which were all conducted remotely (another new feature!), were of NFVi-VNF interoperability combinations that had never been tried before in such open conditions. As a result, not all were successful, but as this is still a very nascent market, it was expected that some of the evaluations were not be completed successfully.

In fact, the success rate -- 25 of the 39 combinations tested, or 64%, passed -- was "a great result," noted EANTC managing director Carsten Rossenhövel.

That higher-than-expected pass rate was due, in part, to the high degree of industry participation: During the seven weeks of testing, 55 engineers from various vendors (including test system partner Ixia) were involved, with support from 14 marketing and communications executives, seven EANTC staff and three from Light Reading/NIA: In total, 79 people were directly involved in the process.

So does that mean that there was a 36% fail rate? It could be viewed that way, but that would be shortsighted. The point of these evaluations is not to identify technology that 'does not work' – it is to see what can be done better, what needs to be fixed. The NIA exists to advance the industry and speed up the process of getting useful technology to market: When the evaluations highlight an interoperability combination that doesn't work, that means the vendors can fix the problems! This can only be good for the industry.

The evaluation also highlighted many other interesting and useful outcomes, not least of which was insight into the performance of OpenStack. For some OpenStack supporters, it is a viable alternative to standardization: "If only everybody would base their implementations on OpenStack, all interop issues would be solved," could be a phrase you might hear in virtualization circles.

Well, according to EANTC's results, OpenStack is far from ready to play an interop role. "There were tons of interop issues despite the fact that all NFVis were based on OpenStack," noted EANTC's Rossenhövel. That's because there are far too many options (including how to manage a VNF), too many versions (with backwards compatibility an oversight), and items left open for vendor consideration (including license management).

So there are many positives to take away from the Phase 1 test results, but there is much, much more to be done: A test combination that passed confirms that the initial lifecycle management tasks work between the parties on a functional basis, but that still leaves many aspects of application-layer functionality, performance, resiliency and orchestration to be evaluated. Those aspects will be the focus of future evaluations.

In the meantime, the following pages will provide you will a full matrix of the test results and detailed insights into the evaluations, the outcomes and the conclusions. So dive in!

Page 2: The Participants

Page 3: Test Setup and Coverage

Page 4: The Results Matrix

Page 5: Interoperability Findings and Challenges

Page 6: Further Interoperability Findings and Challenges

  • Prefer to read this report in a downloadable PDF file? An extended PDF version of this report, which includes individual report cards for each test combination that achieved a 'pass,' is available for free (to any Light Reading registered user) at this link.

    — Ray Le Maistre, Circle me on Google+ Follow me on TwitterVisit my LinkedIn profile, Editor-in-Chief, Light Reading and Carsten Rossenhövel, Managing Director, European Advanced Networking Test Center AG (EANTC) (http://www.eantc.de/), an independent test lab in Berlin. EANTC offers vendor-neutral network test facilities for manufacturers, service providers, and enterprises.

    Next page: The Participants

    (2)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    muzza2011
    50%
    50%
    muzza2011,
    User Rank: Light Beer
    12/10/2015 | 2:39:46 PM
    Network now needs a permanent Proof of Concept lab
    We're at the 'art of the possible' stage, rather than the 'start of the probable' regarding live deployment... which if taken at face value, should wholly sidestep the heinous blunder of productionising a nine 5's infrastructure in for a five 9's expectation.

    As much as the IT proposition to a user involves all layers of the ISO stack, if you screw the network you've lost the farm, which then negates any major investment in state of the art delivery systems up the stack.

    Any tech worth their weight will crash and burn this stuff on Proof of Concept lab... and weld all the fork doors closed... as it'll be their 'nads on the line should if fail in production, not any IT upper heirarchy who decided to make an (unwise) executive decision.

    Fail means *anything* regardless whether its a SNAFU or an hack-attack vector that no-one tested. 

    In short, test to destruction, armour-plate the resultant design, and always have a PoC lab bubbling away in the background for the next iteration, because the genie is now out of the bottle and won't go back in.
    mhhf1ve
    100%
    0%
    mhhf1ve,
    User Rank: Light Sabre
    12/8/2015 | 8:56:52 PM
    Open source doesn't necessarily mean any support available...
    It's always interesting to see how open source platforms still have gaping holes with major unsupported functions. Every fork/flavor of an open source project means more than a few potential gaps for interoperability with other forks.

     
    From The Founder
    The independent evaluation of Nokia's key virtual network functions (VNFs) was a defining moment for the Finnish giant.
    Flash Poll
    Live Streaming Video
    Charting the CSP’s Future
    Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it’s going from two industry veterans.
    LRTV Custom TV
    Cable & the Move to IPTV

    5|31|16   |   04:44   |   (0) comments


    Cable operators are looking at transitioning operations to IPTV, and some have already started. What issues do operators face? What opportunities will this bring to customers? These are just a few of the questions Alticast considers in this interview between Alan Breznick and Susan Crouse at INTX 2016.
    LRTV Custom TV
    Ooredoo at UBBS 2016

    5|31|16   |   06:04   |   (0) comments


    At UBBS 2016 in Dubai, Light Reading is joined by Ooredoo Qatar's Yann Courqueux for an in-depth interview on their success.
    LRTV Custom TV
    du's Success Infrastructure

    5|31|16   |   11:48   |   (0) comments


    At UBBS 2016 in Dubai, Light Reading is joined by du's Jasim Alawadi for an in-depth interview on deploying all types of technology.
    LRTV Custom TV
    What Is Customer Experience Management?

    5|31|16   |   05:53   |   (0) comments


    At UBBS 2016 in Dubai, Light Reading is joined by Huawei's Michael Spratt for an in-depth interview on customer experience management.
    LRTV Custom TV
    UBBS 2016 World Tour Highlight

    5|31|16   |   02:43   |   (0) comments


    At UBBS World Tour 2016 in Dubai, Light Reading takes an exclusive look at the latest Huawei technology and the future of broadband.
    LRTV Interviews
    Getting Somewhere With TV Everywhere

    5|31|16   |   7:08   |   (0) comments


    Comcast's Vito Forlenza describes the progress his company is making with TV Everywhere and the challenges that remain.
    LRTV Custom TV
    Cisco's Innovations in Cable

    5|26|16   |   03:18   |   (0) comments


    Marc Aldrich from Cisco discusses the latest in security, the evolution and momentum for CCAP and what the industry will be seeing next from Cisco.
    LRTV Documentaries
    Leading Lights 2016 Highlights

    5|25|16   |   02:26   |   (1) comment


    Some of the high points from this year's Leading Lights awards dinner at the Hotel Ella in Austin, Texas.
    LRTV Documentaries
    Light Reading Hall of Fame 2016

    5|23|16   |   05:43   |   (0) comments


    Find out who has been welcomed into Light Reading's Hall of Fame this year.
    LRTV Custom TV
    ZTE TM Forum Highlights

    5|23|16   |     |   (0) comments


    ZTE showcased its new ICT solutions at TM Forum in Nice.
    LRTV Interviews
    Gamma's MD on the Emergence of UC2

    5|20|16   |     |   (0) comments


    Gamma Communications Managing Director David Macfarlane believes the unified communications (UC) market has reached a tipping point.
    LRTV Custom TV
    The Ultimate 5-Minute Guide to Digital Customer Engagement

    5|20|16   |     |   (0) comments


    In this short video, you will hear all about how Digital Customer Engagement is the key to meeting customer expectations, keeping them happy, and maximizing revenue. VP Product & Marketing at Pontis, Ofer Razon, breaks down for ...
    Upcoming Live Events
    September 13-14, 2016, The Curtis Hotel, Denver, CO
    December 6-8, 2016,
    June 16-18, 2017, Austin Convention Center, Austin, TX
    All Upcoming Live Events
    Infographics
    A new survey conducted by Heavy Reading and TM Forum shows that CSPs around the world see the move to digital operations as a necessary part of their overall virtualization strategies.
    Hot Topics
    Cisco's Patel Hails 'Microculture' Successes
    Iain Morris, News Editor, 5/26/2016
    Cable Is Eyeing Its Retail Options
    Mari Silbey, Senior Editor, Cable/Video, 5/25/2016
    AT&T's Margaret Chiosi Retires
    Ray Le Maistre, Editor-in-chief, 5/25/2016
    West Coast Gets Its First 10-Gig City
    Mari Silbey, Senior Editor, Cable/Video, 5/26/2016
    IEEE President-Elect Tackles the Digital Divide
    Sarah Thomas, Director, Women in Comms, 5/30/2016
    Like Us on Facebook
    Twitter Feed
    BETWEEN THE CEOs - Executive Interviews
    In this latest installment of the CEO Chat series, Craig Labovitz, co-founder and CEO of Deepfield, sits down with Light Reading's Steve Saunders in Light Reading's New York City office to discuss how Deepfield fits in with the big data trend and more.
    Grant van Rooyen, president and CEO of Cologix, sits down with Steve Saunders, founder and CEO of Light Reading, in the vendor's New Jersey facility to offer an inside look at the company's success story and discuss the importance of security in the telecom industry.
    Animals with Phones
    Please Pass the Waterproof Phone Click Here
    Walruses: more tech-savvy than you might think.
    Live Digital Audio

    Our world has evolved through innovation from the Industrial Revolution of the 1740s to the information age, and it is now entering the Fourth Industrial Revolution, driven by technology. Technology is driving a paradigm shift in the way digital solutions deliver a connected world, changing the way we live, communicate and provide solutions. It can have a powerful impact on how we tackle some of the world’s most pressing problems. In this radio show, Caroline Dowling, President of Communications Infrastructure & Enterprise Computing at Flex, will join Women in Comms Director Sarah Thomas to discuss the impact technology has on society and how it can be a game-changer across the globe; improving lives and creating a smarter world. Dowling, a Cork, Ireland, native and graduate of Harvard Business School's Advanced Management Program, will also discuss her experience managing an international team focused on innovation in an age of high-speed change.