& cplSiteName &

EXCLUSIVE! NFV Interop Evaluation Results

Light Reading
Prime Reading
Light Reading
12/8/2015
100%
0%

Of all the industry test results out there, this is the one that people have been waiting for.

In a historic first for the communications networking industry, Light Reading, in partnership with EANTC on behalf of the New IP Agency (NIA), is happy and proud to announce the publication of the world's first independent interoperability evaluation of NFV infrastructure, focused (in Phase 1) on multivendor NFV infrastructure-to-virtual network function (VNF) interoperability.

These results are just a first step on a very important journey for an industry that is in dramatic upheaval. Never before have network operators, faced with the unknown and daunting challenges of introducing virtualized functions and their supporting infrastructure platforms, needed independent test results as much as they need them now: Such independent evaluations can cut down development times, save operators resources, and help speed up further tests, field trials and live deployments.

That's why we have been moving fast. The Phase 1 evaluation, which was publicized, set up and completed in just a few months (an extraordinarily short time for such a procedure), included 12 vendors, which between them submitted four NFV infrastructure (NFVi) platforms and 17 VNFs.

The NFVi vendors, which all offered platforms based on OpenStack, were:

  • Alcatel-Lucent
  • Cisco Systems
  • Huawei Technologies
  • Juniper Networks

The VNF vendors were:

  • Alcatel-Lucent
  • Cisco Systems
  • Cobham Wireless
  • Hitachi Communication Technologies America
  • Huawei Technologies
  • IneoQuest Technologies
  • Juniper Networks
  • Metaswitch Networks
  • NetNumber
  • Netrounds
  • Procera Networks
  • Sonus Networks

The VNFs came in many shapes and forms, from virtual routers, EPC (evolved packet core) and IMS components, firewalls, test probes, deep packet inspection (DPI) modules and session border controllers.

The testing focused on the functional interoperability of NFVi platforms and various VNFs, but did not include the sort of tests more commonly associated with communications infrastructure, such as performance, scalability and resilience testing: In addition, these tests did not include any management and orchestration (MANO) evaluations (they will be the focus of a later evaluation phase).

The tests carried out by the EANTC team, which were all conducted remotely (another new feature!), were of NFVi-VNF interoperability combinations that had never been tried before in such open conditions. As a result, not all were successful, but as this is still a very nascent market, it was expected that some of the evaluations were not be completed successfully.

In fact, the success rate -- 25 of the 39 combinations tested, or 64%, passed -- was "a great result," noted EANTC managing director Carsten Rossenhövel.

That higher-than-expected pass rate was due, in part, to the high degree of industry participation: During the seven weeks of testing, 55 engineers from various vendors (including test system partner Ixia) were involved, with support from 14 marketing and communications executives, seven EANTC staff and three from Light Reading/NIA: In total, 79 people were directly involved in the process.

So does that mean that there was a 36% fail rate? It could be viewed that way, but that would be shortsighted. The point of these evaluations is not to identify technology that 'does not work' – it is to see what can be done better, what needs to be fixed. The NIA exists to advance the industry and speed up the process of getting useful technology to market: When the evaluations highlight an interoperability combination that doesn't work, that means the vendors can fix the problems! This can only be good for the industry.

The evaluation also highlighted many other interesting and useful outcomes, not least of which was insight into the performance of OpenStack. For some OpenStack supporters, it is a viable alternative to standardization: "If only everybody would base their implementations on OpenStack, all interop issues would be solved," could be a phrase you might hear in virtualization circles.

Well, according to EANTC's results, OpenStack is far from ready to play an interop role. "There were tons of interop issues despite the fact that all NFVis were based on OpenStack," noted EANTC's Rossenhövel. That's because there are far too many options (including how to manage a VNF), too many versions (with backwards compatibility an oversight), and items left open for vendor consideration (including license management).

So there are many positives to take away from the Phase 1 test results, but there is much, much more to be done: A test combination that passed confirms that the initial lifecycle management tasks work between the parties on a functional basis, but that still leaves many aspects of application-layer functionality, performance, resiliency and orchestration to be evaluated. Those aspects will be the focus of future evaluations.

In the meantime, the following pages will provide you will a full matrix of the test results and detailed insights into the evaluations, the outcomes and the conclusions. So dive in!

Page 2: The Participants

Page 3: Test Setup and Coverage

Page 4: The Results Matrix

Page 5: Interoperability Findings and Challenges

Page 6: Further Interoperability Findings and Challenges

  • Prefer to read this report in a downloadable PDF file? An extended PDF version of this report, which includes individual report cards for each test combination that achieved a 'pass,' is available for free (to any Light Reading registered user) at this link.

    — Ray Le Maistre, Circle me on Google+ Follow me on TwitterVisit my LinkedIn profile, Editor-in-Chief, Light Reading and Carsten Rossenhövel, Managing Director, European Advanced Networking Test Center AG (EANTC) (http://www.eantc.de/), an independent test lab in Berlin. EANTC offers vendor-neutral network test facilities for manufacturers, service providers, and enterprises.

    Next page: The Participants

    (2)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    muzza2011
    50%
    50%
    muzza2011,
    User Rank: Light Beer
    12/10/2015 | 2:39:46 PM
    Network now needs a permanent Proof of Concept lab
    We're at the 'art of the possible' stage, rather than the 'start of the probable' regarding live deployment... which if taken at face value, should wholly sidestep the heinous blunder of productionising a nine 5's infrastructure in for a five 9's expectation.

    As much as the IT proposition to a user involves all layers of the ISO stack, if you screw the network you've lost the farm, which then negates any major investment in state of the art delivery systems up the stack.

    Any tech worth their weight will crash and burn this stuff on Proof of Concept lab... and weld all the fork doors closed... as it'll be their 'nads on the line should if fail in production, not any IT upper heirarchy who decided to make an (unwise) executive decision.

    Fail means *anything* regardless whether its a SNAFU or an hack-attack vector that no-one tested. 

    In short, test to destruction, armour-plate the resultant design, and always have a PoC lab bubbling away in the background for the next iteration, because the genie is now out of the bottle and won't go back in.
    mhhf1ve
    100%
    0%
    mhhf1ve,
    User Rank: Light Sabre
    12/8/2015 | 8:56:52 PM
    Open source doesn't necessarily mean any support available...
    It's always interesting to see how open source platforms still have gaping holes with major unsupported functions. Every fork/flavor of an open source project means more than a few potential gaps for interoperability with other forks.

     
    Light Reading’s Upskill U is a FREE, interactive, online educational resource that delivers must-have education on themes that relate to the overall business transformation taking place in the communications industry.
    NEXT COURSE
    Wednesday, August 3, 1:00PM EDT
    The Central Office Re-Architected as a Data Center
    Guru Parulkar, Executive Director, Open Networking Research Center, Open Networking Lab
    UPCOMING COURSE SCHEDULE
    Wednesday, August 10, 1:00PM EDT
    Telcos & Open Source 101
    Phil Robb, Senior Technical Director, OpenDaylight
    Friday, August 12, 1:00PM EDT
    The Role of Open Source in NFV
    Jim Fagan, Director, Cloud Practice, Telstra
    Wednesday, August 17, 1:00PM EDT
    Using Open Source for Data Centers and Cloud Services
    Roz Roseboro, Senior Analyst, Heavy Reading
    in association with:
    From The Founder
    Light Reading today starts a new voyage as part of a larger Enterprise.
    Flash Poll
    Live Streaming Video
    Charting the CSP's Future
    Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it's going from two industry veterans.
    Women in Comms Introduction Videos
    Fujitsu Sales Leader Shares Lessons Learned

    7|27|16   |   5:12   |   (0) comments


    As Fujitsu's only female sales leader, Annie Bogue knows the importance of asking for what you want, being flexible (she's been relocated five times), keeping a meticulous calendar, 'leaning in,' working harder than everyone else around you, being aware and more.
    Telecom Innovators Video Showcase
    VeEX Test & Measurement Solutions

    7|25|16   |   08:57   |   (0) comments


    Cyrille Morelle, president and CEO of VeEX Inc., talks test and measurement with Light Reading's Steve Saunders at BCE 2016. This includes innovative products such as VeSion Cloud-Based platform for network monitoring; MTTplus Modular Test platform for Access, Business, Carrier Ethernet, Transport and Core services; and OPX-BOX+ for Fiber Optics.
    LRTV Custom TV
    VeEX: Live From BCE 2016

    7|25|16   |   03:20   |   (0) comments


    VeEX's Senior Director of Business Development, Perry Romano, explains how VeEX provides tools to help install, maintain, monitor and manage network infrastructure efficiently and effectively. The portfolio of products on display include the RXT-6000, MTTplus and TX300s.
    LRTV Custom TV
    Real-Time Telemetry & Analytics for Intelligent SDN Orchestration

    7|25|16   |   03:09   |   (0) comments


    Packet Design CEO Scott Sherwood discusses how real-time network telemetry and analytics are enabling a new breed of SDN orchestration applications.
    From the Founder
    The Russo Report: Driving Disruption

    7|25|16   |   07:44   |   (0) comments


    In the first episode of a four-part series, Light Reading Founder and CEO Steve Saunders and Calix President and CEO Carl Russo drive around town discussing the disruptive mega-changes in the communications industry and where hope lies for service providers to meet the escalating demands of the cloud.
    LRTV Custom TV
    NetScout: Maximizing Enterprise Cloud for Digital Transformation

    7|20|16   |   04:53   |   (0) comments


    Light Reading Editor Mitch Wagner talks to NetScout CMO Jim McNiel about maximizing the benefits of enterprise cloud and digital transformation while minimizing potential pitfalls with a proper monitoring and instrumentation strategy.
    Women in Comms Introduction Videos
    Ciena's VP Offers a Career Crash Course

    7|20|16   |   4:14   |   (2) comments


    How did Ciena's Vice President of Sales, Angela Finn, carve out her career path? Simple, she tells WiC. She stayed true to her company, customers and principles. She shares her advice for women on how to be authentic and credible, as well as for companies that want to make a real change to their culture and practices.
    LRTV Custom TV
    NFV in 2016: Part 2 – Climbing the Virtualization Maturity Curve

    7|19|16   |   06:56   |   (0) comments


    Many of the initial use case implementations are single-vendor and self-contained. The industry is still climbing the virtualization maturity curve, needing further clarity and stability in the NFV infrastructure (NFVi) and greater availability and choice of virtualized network functions (VNFs). Interoperability between NFVis and VNFs from different vendors ...
    Telecom Innovators Video Showcase
    Versa Networks' Kumar Mehta on SD-WAN Managed Services

    7|19|16   |     |   (0) comments


    In Silicon Valley, Steve Saunders sits down with Versa's Kumar Mehta for an interview focused on why service providers are building SD-WAN managed services, and how Versa's telco customers are innovating.
    LRTV Custom TV
    Juniper Networks & The Evolution of NFV

    7|19|16   |   06:01   |   (0) comments


    Senior Juniper Networks executives talk to Light Reading Founder & CEO Steve Saunders about NFV developments and the recent independent evaluation by test lab EANTC of Juniper's Cloud CPE solution.
    LRTV Interviews
    CenturyLink Goes Beyond Managed WiFi

    7|19|16   |     |   (0) comments


    CenturyLink's managed WiFi allows enterprises, such as retailers and resorts, to track guest WiFi usage in order to help them better communicate with customers.
    LRTV Interviews
    AT&T Launches Network Functions on Demand

    7|17|16   |   05:26   |   (0) comments


    Roman Pacewicz, Senior Vice President, Offer Management & Service Integration, AT&T Business Solutions, discusses the operator's launch of its Network Functions on Demand service.
    Upcoming Live Events
    September 13-14, 2016, The Curtis Hotel, Denver, CO
    September 27, 2016, Philadelphia, PA
    November 3, 2016, The Montcalm Marble Arch, London
    November 30, 2016, The Westin Times Square, New York City
    December 6-8, 2016,
    May 16-17, 2017, Austin Convention Center, Austin, TX
    All Upcoming Live Events
    Infographics
    Five of the Top 10 most targeted countries in Check Point Software Technologies' global Malware & Threat Index for Q1 2016 are in Africa.
    Hot Topics
    Verizon Sports Big Plans for Yahoo
    Alan Breznick, Cable/Video Practice Leader, Light Reading, 7/26/2016
    Yahoo Signing Off in $4.83B Sale to Verizon
    Mari Silbey, Senior Editor, Cable/Video, 7/25/2016
    Ericsson Board Has Been Asleep at the Wheel – Consultant
    Ray Le Maistre, Editor-in-chief, 7/25/2016
    Ericsson Ejects CEO Vestberg
    Ray Le Maistre, Editor-in-chief, 7/25/2016
    Is Dish Going Down the Drain?
    Alan Breznick, Cable/Video Practice Leader, Light Reading, 7/21/2016
    Like Us on Facebook
    Twitter Feed
    BETWEEN THE CEOs - Executive Interviews
    There's no question that, come 2020, 5G technology will turn the world's conception of what mobile networking is on its head. Within the world of 5G development, Dr. ...
    I've enjoyed interviewing many interesting people since I rejoined Light Reading, but William A. "Bill" Owens certainly takes the biscuit, as we say where I come from.
    Animals with Phones
    Live Digital Audio

    Our world has evolved through innovation from the Industrial Revolution of the 1740s to the information age, and it is now entering the Fourth Industrial Revolution, driven by technology. Technology is driving a paradigm shift in the way digital solutions deliver a connected world, changing the way we live, communicate and provide solutions. It can have a powerful impact on how we tackle some of the world’s most pressing problems. In this radio show, Caroline Dowling, President of Communications Infrastructure & Enterprise Computing at Flex, will join Women in Comms Director Sarah Thomas to discuss the impact technology has on society and how it can be a game-changer across the globe; improving lives and creating a smarter world. Dowling, a Cork, Ireland, native and graduate of Harvard Business School's Advanced Management Program, will also discuss her experience managing an international team focused on innovation in an age of high-speed change.