& cplSiteName &

EXCLUSIVE! NFV Interop Evaluation Results

Light Reading
Prime Reading
Light Reading
12/8/2015
100%
0%

Of all the industry test results out there, this is the one that people have been waiting for.

In a historic first for the communications networking industry, Light Reading, in partnership with EANTC on behalf of the New IP Agency (NIA), is happy and proud to announce the publication of the world's first independent interoperability evaluation of NFV infrastructure, focused (in Phase 1) on multivendor NFV infrastructure-to-virtual network function (VNF) interoperability.

These results are just a first step on a very important journey for an industry that is in dramatic upheaval. Never before have network operators, faced with the unknown and daunting challenges of introducing virtualized functions and their supporting infrastructure platforms, needed independent test results as much as they need them now: Such independent evaluations can cut down development times, save operators resources, and help speed up further tests, field trials and live deployments.

That's why we have been moving fast. The Phase 1 evaluation, which was publicized, set up and completed in just a few months (an extraordinarily short time for such a procedure), included 12 vendors, which between them submitted four NFV infrastructure (NFVi) platforms and 17 VNFs.

The NFVi vendors, which all offered platforms based on OpenStack, were:

  • Alcatel-Lucent
  • Cisco Systems
  • Huawei Technologies
  • Juniper Networks

The VNF vendors were:

  • Alcatel-Lucent
  • Cisco Systems
  • Cobham Wireless
  • Hitachi Communication Technologies America
  • Huawei Technologies
  • IneoQuest Technologies
  • Juniper Networks
  • Metaswitch Networks
  • NetNumber
  • Netrounds
  • Procera Networks
  • Sonus Networks

The VNFs came in many shapes and forms, from virtual routers, EPC (evolved packet core) and IMS components, firewalls, test probes, deep packet inspection (DPI) modules and session border controllers.

The testing focused on the functional interoperability of NFVi platforms and various VNFs, but did not include the sort of tests more commonly associated with communications infrastructure, such as performance, scalability and resilience testing: In addition, these tests did not include any management and orchestration (MANO) evaluations (they will be the focus of a later evaluation phase).

The tests carried out by the EANTC team, which were all conducted remotely (another new feature!), were of NFVi-VNF interoperability combinations that had never been tried before in such open conditions. As a result, not all were successful, but as this is still a very nascent market, it was expected that some of the evaluations were not be completed successfully.

In fact, the success rate -- 25 of the 39 combinations tested, or 64%, passed -- was "a great result," noted EANTC managing director Carsten Rossenhövel.

That higher-than-expected pass rate was due, in part, to the high degree of industry participation: During the seven weeks of testing, 55 engineers from various vendors (including test system partner Ixia) were involved, with support from 14 marketing and communications executives, seven EANTC staff and three from Light Reading/NIA: In total, 79 people were directly involved in the process.

So does that mean that there was a 36% fail rate? It could be viewed that way, but that would be shortsighted. The point of these evaluations is not to identify technology that 'does not work' – it is to see what can be done better, what needs to be fixed. The NIA exists to advance the industry and speed up the process of getting useful technology to market: When the evaluations highlight an interoperability combination that doesn't work, that means the vendors can fix the problems! This can only be good for the industry.

The evaluation also highlighted many other interesting and useful outcomes, not least of which was insight into the performance of OpenStack. For some OpenStack supporters, it is a viable alternative to standardization: "If only everybody would base their implementations on OpenStack, all interop issues would be solved," could be a phrase you might hear in virtualization circles.

Well, according to EANTC's results, OpenStack is far from ready to play an interop role. "There were tons of interop issues despite the fact that all NFVis were based on OpenStack," noted EANTC's Rossenhövel. That's because there are far too many options (including how to manage a VNF), too many versions (with backwards compatibility an oversight), and items left open for vendor consideration (including license management).

So there are many positives to take away from the Phase 1 test results, but there is much, much more to be done: A test combination that passed confirms that the initial lifecycle management tasks work between the parties on a functional basis, but that still leaves many aspects of application-layer functionality, performance, resiliency and orchestration to be evaluated. Those aspects will be the focus of future evaluations.

In the meantime, the following pages will provide you will a full matrix of the test results and detailed insights into the evaluations, the outcomes and the conclusions. So dive in!

Page 2: The Participants

Page 3: Test Setup and Coverage

Page 4: The Results Matrix

Page 5: Interoperability Findings and Challenges

Page 6: Further Interoperability Findings and Challenges

  • Prefer to read this report in a downloadable PDF file? An extended PDF version of this report, which includes individual report cards for each test combination that achieved a 'pass,' is available for free (to any Light Reading registered user) at this link.

    — Ray Le Maistre, Circle me on Google+ Follow me on TwitterVisit my LinkedIn profile, Editor-in-Chief, Light Reading and Carsten Rossenhövel, Managing Director, European Advanced Networking Test Center AG (EANTC) (http://www.eantc.de/), an independent test lab in Berlin. EANTC offers vendor-neutral network test facilities for manufacturers, service providers, and enterprises.

    Next page: The Participants

    (2)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    muzza2011
    50%
    50%
    muzza2011,
    User Rank: Light Beer
    12/10/2015 | 2:39:46 PM
    Network now needs a permanent Proof of Concept lab
    We're at the 'art of the possible' stage, rather than the 'start of the probable' regarding live deployment... which if taken at face value, should wholly sidestep the heinous blunder of productionising a nine 5's infrastructure in for a five 9's expectation.

    As much as the IT proposition to a user involves all layers of the ISO stack, if you screw the network you've lost the farm, which then negates any major investment in state of the art delivery systems up the stack.

    Any tech worth their weight will crash and burn this stuff on Proof of Concept lab... and weld all the fork doors closed... as it'll be their 'nads on the line should if fail in production, not any IT upper heirarchy who decided to make an (unwise) executive decision.

    Fail means *anything* regardless whether its a SNAFU or an hack-attack vector that no-one tested. 

    In short, test to destruction, armour-plate the resultant design, and always have a PoC lab bubbling away in the background for the next iteration, because the genie is now out of the bottle and won't go back in.
    mhhf1ve
    100%
    0%
    mhhf1ve,
    User Rank: Light Sabre
    12/8/2015 | 8:56:52 PM
    Open source doesn't necessarily mean any support available...
    It's always interesting to see how open source platforms still have gaping holes with major unsupported functions. Every fork/flavor of an open source project means more than a few potential gaps for interoperability with other forks.

     
    Light Reading’s Upskill U is a FREE, interactive, online educational resource that delivers must-have education on themes that relate to the overall business transformation taking place in the communications industry.
    LIVE NOW!
    Friday, December 2, 1:00PM EST
    The SDN Approach to IP & Optical Integration
    Sterling Perrin, Senior Analyst, Heavy Reading
    UPCOMING COURSE SCHEDULE
    Friday, December 2, 1:00PM EST
    The SDN Approach to IP & Optical Integration
    Sterling Perrin, Senior Analyst, Heavy Reading
    in association with:
    From The Founder
    Light Reading today starts a new voyage as part of a larger Enterprise.
    Flash Poll
    Live Streaming Video
    Charting the CSP's Future
    Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it's going from two industry veterans.
    Women in Comms Introduction Videos
    Korn Ferry Consultant: How to Find, Cultivate & Be the Best Talent

    11|30|16   |   4:10   |   (1) comment


    Erin Callaghan, a managing consultant for Korn Ferry Futurestep, shares strategies for companies to improve how they recruit and for women to ensure they don't get lost in the pipeline.
    LRTV Custom TV
    We Can Make the World More Sustainable

    11|29|16   |     |   (0) comments


    GeSI is a global e-Sustainability Initiative organization bringing together 40 big multinational companies around the world. According to GeSI's report, information and communication technology can make the world more sustainable. Luis Neves, chairman of GeSI, shared with us his opinion at Ultra-broadband Forum (UBBF2016).
    LRTV Custom TV
    Finding a New Way to Engage Customers & Drive Revenue

    11|29|16   |     |   (0) comments


    Mobile revenues are declining. Digicel, a player in the Caribbean telecommunications/entertainment space, has found a new way to engage customers and drive revenue. John Quinn, CTO of Digicel, shared with us its story at Ultra-broadband Forum (UBBF2016)
    LRTV Custom TV
    Do You Really Need Gigabit Infrastructure?

    11|29|16   |     |   (0) comments


    Altibox is the biggest fiber-to-the-home (FTTH) player and the largest provider of video and TV in Norway. They started out with zero customers in 2002. Now they have close to half a million households and companies attached to their FTTH business. Nils Arne, CEO of Altibox shared with us their story and insight on 5G at Ultra-broadband Forum (UBBF2016).
    LRTV Custom TV
    BT’s Openreach Strategy & Its Updates in 2016

    11|29|16   |     |   (0) comments


    A lot of developments at Openreach this year in terms of strategy and planned investments. Peter Bell, CIO of Openreach BT, shared with us the updates of Openreach at Ultra-broadband Forum (UBBF2016).
    LRTV Custom TV
    ITU: The Broadband Is Our Future

    11|29|16   |     |   (0) comments


    At Ultra-broadband Forum, Houlin Zhao, Secretary General of ITU, discussed how important it is for countries, companies and everybody to be working together to help to build the broadband and digital economies (UBBF2016).
    LRTV Custom TV
    Tackling 5G in Dallas

    11|28|16   |     |   (0) comments


    Here are our highlights of the 5G North America show in Dallas, Texas with Light Reading's Dan Jones.
    LRTV Interviews
    Cox Prepping for Virtualization Trials

    11|14|16   |     |   (0) comments


    In this video interview, Cox's Jeff Finkelstein discusses MSO's plans to test managed business services in early 2017 and tackle Distributed Access Architectures.
    LRTV Custom TV
    Drivers & Potential of NGP

    11|11|16   |     |   (0) comments


    ETSI has created an Industry Specification Group to work on Next Generation Protocols (NGP ISG), looking at evolving communications and networking protocols to provide the scale, security, mobility and ease of deployment required for the connected society of the 21st century. The NGP ISG will identify the requirements for next generation protocols and network ...
    LRTV Custom TV
    Huawei IP 2020 for Future Networks

    11|11|16   |     |   (0) comments


    Future Networks should satisfy many requirements such as high throughput, extremely low latency, flexible mobility, intrinsic security, networking automation, and so forth. The Chief Architect of Huawei Future Networks addresses a holistic solution, i.e., IP 2020, to achieve these requirements for various future life scenarios (e.g., autonomous driving, tactile ...
    LRTV Custom TV
    Digital Object Architecture

    11|11|16   |     |   (0) comments


    Digital Object Architecture provides a basic information infrastructure that can facilitate interoperability between or among different systems, processes, and other information resources, including different identity management systems. Digital objects are networked objects that are named by digital object identifiers and instantiated by an infrastructure service ...
    LRTV Custom TV
    BT's Openreach Has High Hopes for Long-Reach VDSL

    11|11|16   |   06:04   |   (0) comments


    Peter Bell, Network Portfolio CIO at BT's access business Openreach, talks about the operator's trial of a new broadband access technology called Long Reach VDSL.
    Upcoming Live Events
    December 6-8, 2016, The Westin Excelsior, Rome
    May 16-17, 2017, Austin Convention Center, Austin, TX
    All Upcoming Live Events
    Infographics
    Hot Topics
    AT&T Debuts DirecTV Now on New Video Platform
    Mari Silbey, Senior Editor, Cable/Video, 11/28/2016
    Apple Seeds 5G? Seeks 'Multi-Gigabit' Chip Designer
    Dan Jones, Mobile Editor, 11/30/2016
    Altice Plans FTTH for Entire US Footprint
    Iain Morris, News Editor, 11/30/2016
    Altice FTTH Bill Could Hit Almost $9.6B in US
    Iain Morris, News Editor, 12/1/2016
    Samsung Bows to Investors, Considers Revamp
    Iain Morris, News Editor, 11/29/2016
    Like Us on Facebook
    Twitter Feed
    BETWEEN THE CEOs - Executive Interviews
    Eyal Waldman, CEO of Mellanox Technologies, speaks to Steve Saunders, CEO of Light Reading, for an exclusive interview about the 100 GB cable challenge, cybersecurity and much more.
    Join us for an in-depth interview between Steve Saunders of Light Reading and Alexis Black Bjorlin of Intel as they discuss the release of the company's Silicon Photonics platform, its performance, long-term prospects, customer expectations and much more.
    Live Digital Audio

    Even when there's a strong pipeline of female talent in the comms industry, it tends to leak all the way to the top. McKinsey & Company says women experience pipeline leakage at three primary points: being unable to enter, being stuck in the middle or being locked out of the top. Each pipeline pain point presents its own challenges, but also opportunities to stop the leak. Wireless operator Sprint is making a conscious effort to improve its own pipeline from new recruits to the C-suite, and it wants the rest of the industry to do the same. In this Women in Comms radio show, WiC Board Member and Sprint Vice President of Enterprise Sales Nelly Pitocco will give us her take on the industry's pipeline challenges. Pitocco, who joined Sprint in May and has spent 20 years in the comms industry, will also offer solutions, share how Sprint is tackling the challenge within its own organization and take your questions live on air.