& cplSiteName &

EXCLUSIVE! NFV Interop Evaluation Results

Light Reading
Prime Reading
Light Reading
12/8/2015
100%
0%

Of all the industry test results out there, this is the one that people have been waiting for.

In a historic first for the communications networking industry, Light Reading, in partnership with EANTC on behalf of the New IP Agency (NIA), is happy and proud to announce the publication of the world's first independent interoperability evaluation of NFV infrastructure, focused (in Phase 1) on multivendor NFV infrastructure-to-virtual network function (VNF) interoperability.

These results are just a first step on a very important journey for an industry that is in dramatic upheaval. Never before have network operators, faced with the unknown and daunting challenges of introducing virtualized functions and their supporting infrastructure platforms, needed independent test results as much as they need them now: Such independent evaluations can cut down development times, save operators resources, and help speed up further tests, field trials and live deployments.

That's why we have been moving fast. The Phase 1 evaluation, which was publicized, set up and completed in just a few months (an extraordinarily short time for such a procedure), included 12 vendors, which between them submitted four NFV infrastructure (NFVi) platforms and 17 VNFs.

The NFVi vendors, which all offered platforms based on OpenStack, were:

  • Alcatel-Lucent
  • Cisco Systems
  • Huawei Technologies
  • Juniper Networks

The VNF vendors were:

  • Alcatel-Lucent
  • Cisco Systems
  • Cobham Wireless
  • Hitachi Communication Technologies America
  • Huawei Technologies
  • IneoQuest Technologies
  • Juniper Networks
  • Metaswitch Networks
  • NetNumber
  • Netrounds
  • Procera Networks
  • Sonus Networks

The VNFs came in many shapes and forms, from virtual routers, EPC (evolved packet core) and IMS components, firewalls, test probes, deep packet inspection (DPI) modules and session border controllers.

The testing focused on the functional interoperability of NFVi platforms and various VNFs, but did not include the sort of tests more commonly associated with communications infrastructure, such as performance, scalability and resilience testing: In addition, these tests did not include any management and orchestration (MANO) evaluations (they will be the focus of a later evaluation phase).

The tests carried out by the EANTC team, which were all conducted remotely (another new feature!), were of NFVi-VNF interoperability combinations that had never been tried before in such open conditions. As a result, not all were successful, but as this is still a very nascent market, it was expected that some of the evaluations were not be completed successfully.

In fact, the success rate -- 25 of the 39 combinations tested, or 64%, passed -- was "a great result," noted EANTC managing director Carsten Rossenhövel.

That higher-than-expected pass rate was due, in part, to the high degree of industry participation: During the seven weeks of testing, 55 engineers from various vendors (including test system partner Ixia) were involved, with support from 14 marketing and communications executives, seven EANTC staff and three from Light Reading/NIA: In total, 79 people were directly involved in the process.

So does that mean that there was a 36% fail rate? It could be viewed that way, but that would be shortsighted. The point of these evaluations is not to identify technology that 'does not work' – it is to see what can be done better, what needs to be fixed. The NIA exists to advance the industry and speed up the process of getting useful technology to market: When the evaluations highlight an interoperability combination that doesn't work, that means the vendors can fix the problems! This can only be good for the industry.

The evaluation also highlighted many other interesting and useful outcomes, not least of which was insight into the performance of OpenStack. For some OpenStack supporters, it is a viable alternative to standardization: "If only everybody would base their implementations on OpenStack, all interop issues would be solved," could be a phrase you might hear in virtualization circles.

Well, according to EANTC's results, OpenStack is far from ready to play an interop role. "There were tons of interop issues despite the fact that all NFVis were based on OpenStack," noted EANTC's Rossenhövel. That's because there are far too many options (including how to manage a VNF), too many versions (with backwards compatibility an oversight), and items left open for vendor consideration (including license management).

So there are many positives to take away from the Phase 1 test results, but there is much, much more to be done: A test combination that passed confirms that the initial lifecycle management tasks work between the parties on a functional basis, but that still leaves many aspects of application-layer functionality, performance, resiliency and orchestration to be evaluated. Those aspects will be the focus of future evaluations.

In the meantime, the following pages will provide you will a full matrix of the test results and detailed insights into the evaluations, the outcomes and the conclusions. So dive in!

Page 2: The Participants

Page 3: Test Setup and Coverage

Page 4: The Results Matrix

Page 5: Interoperability Findings and Challenges

Page 6: Further Interoperability Findings and Challenges

  • Prefer to read this report in a downloadable PDF file? An extended PDF version of this report, which includes individual report cards for each test combination that achieved a 'pass,' is available for free (to any Light Reading registered user) at this link.

    — Ray Le Maistre, Circle me on Google+ Follow me on TwitterVisit my LinkedIn profile, Editor-in-Chief, Light Reading and Carsten Rossenhövel, Managing Director, European Advanced Networking Test Center AG (EANTC) (http://www.eantc.de/), an independent test lab in Berlin. EANTC offers vendor-neutral network test facilities for manufacturers, service providers, and enterprises.

    Next page: The Participants

    (2)  | 
    Comment  | 
    Print  | 
  • Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
    muzza2011
    50%
    50%
    muzza2011,
    User Rank: Light Beer
    12/10/2015 | 2:39:46 PM
    Network now needs a permanent Proof of Concept lab
    We're at the 'art of the possible' stage, rather than the 'start of the probable' regarding live deployment... which if taken at face value, should wholly sidestep the heinous blunder of productionising a nine 5's infrastructure in for a five 9's expectation.

    As much as the IT proposition to a user involves all layers of the ISO stack, if you screw the network you've lost the farm, which then negates any major investment in state of the art delivery systems up the stack.

    Any tech worth their weight will crash and burn this stuff on Proof of Concept lab... and weld all the fork doors closed... as it'll be their 'nads on the line should if fail in production, not any IT upper heirarchy who decided to make an (unwise) executive decision.

    Fail means *anything* regardless whether its a SNAFU or an hack-attack vector that no-one tested. 

    In short, test to destruction, armour-plate the resultant design, and always have a PoC lab bubbling away in the background for the next iteration, because the genie is now out of the bottle and won't go back in.
    mhhf1ve
    100%
    0%
    mhhf1ve,
    User Rank: Light Sabre
    12/8/2015 | 8:56:52 PM
    Open source doesn't necessarily mean any support available...
    It's always interesting to see how open source platforms still have gaping holes with major unsupported functions. Every fork/flavor of an open source project means more than a few potential gaps for interoperability with other forks.

     
    From The Founder
    Kicking off BCE 2017, Light Reading founder Steve Saunders lays blame for NFV's slow ramp-up and urges telecom to return to old-fashioned standards building and interoperability.
    Flash Poll
    Live Streaming Video
    Charting the CSP's Future
    Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it's going from two industry veterans.
    LRTV Interviews
    CenturyLink: Let's Get Past SD-WAN Hype

    6|23|17   |   04:02   |   (0) comments


    Technology becomes a "shiny object" unless it's properly focused on solving business needs for enterprise customers, says Bill Grubbs, network solutions architect for CenturyLink. He explains to Light Reading why SD-WAN deployments have to be tailored to specific needs – and more.
    Women in Comms Introduction Videos
    Infinera's Sales Director Paints Tech's Big Picture

    6|21|17   |   4:14   |   (1) comment


    Shannon Williams, Infinera's director of sales, shares how she achieves work's many balancing acts -- between her role and the broader company, today and tomorrow's tech and more.
    LRTV Custom TV
    SD-WAN Innovation & Trends

    6|20|17   |     |   (0) comments


    Versa CEO Kelly Ahuja discusses with Carol Wilson the current status and trends in the SD-WAN market, Versa's innovation around building a software platform with broad contextualization, and the advantages that startups can bring to the SD-WAN market.
    LRTV Interviews
    Ovum's Dario Talmesio on 5G in Europe

    6|20|17   |   02:16   |   (0) comments


    At 5G World 2017, Dario Talmesio, principal analyst and practice leader on Ovum's fixed and mobile telecoms European team, explains the emerging trends amongst European operators as they prepare for 5G.
    LRTV Custom TV
    Putting Power on a Pedestal

    6|19|17   |     |   (0) comments


    ARRIS's John Ulm says a major accomplishment of SCTE•ISBE's Energy 2020 program is increased focus on power cost and consumption, including inclusion of energy requirements in operators' RFPs and RFIs.
    LRTV Custom TV
    Gigabit Access: The Last-Mile Pipe for All Future Services

    6|19|17   |     |   (0) comments


    A Gigabit access platform being deployed today must be able to deliver all types of services to an increasing number of devices. A non-blocking architecture is necessary to support the ever-increasing growth in bandwidth demand. The Huawei Gigabit access solution is based on a distributed design that is fully scalable to deliver a unprecedented performance.
    LRTV Custom TV
    Key Factors to Successfully Deploy an SD-WAN Service

    6|19|17   |     |   (0) comments


    As service providers transition their SD-WAN solution from trials and limited deployments into production at large scale, there are important considerations to successfully operationalize these solutions and realize their full potential, without adding complexity, introducing uncertainty or disrupting current business operations. Sunil Khandekar, CEO and Founder ...
    LRTV Custom TV
    IoT Solutions: Rational Exuberance

    6|19|17   |     |   (0) comments


    IoT solutions are morphing from hype into viable business opportunities. Huawei has the platform and ecosystem support to help carriers successfully address new business opportunities in the IoT space.
    LRTV Custom TV
    Realizing ICN as a Network Slice for Mobile Data Distribution

    6|19|17   |     |   (1) comment


    Network slicing in 5G allows the potential introduction of new network architectures such as Information-centric Networks (ICN) as a slice, managed over a shared pool of compute, storage and bandwidth resource. Services over an ICN slice can benefit from many architectural features such as Name Based Networking, Security, Multicasting, Multi-homing, Mobility, ...
    LRTV Interviews
    Ovum's Mike Roberts on 5G Uptake

    6|19|17   |   04:08   |   (0) comments


    Mike Roberts, research director for Ovum's service provider markets group, explains why he has boosted his 5G subscriptions forecast.
    LRTV Interviews
    AT&T's Hubbard on Intersection of SD-WAN & MPLS

    6|15|17   |     |   (0) comments


    Rick Hubbard, SVP of Network Product Management for AT&T Business Solutions, discusses how AT&T's approach to SD-WAN fits in with its overall virtualization strategy, explains how SD-WAN can improve enterprise customers' use of the cloud and addresses the intersection of SD-WAN and MPLS.
    Telecom Innovators Video Showcase
    Keep Connected IoT Devices Under Control With Allot

    6|15|17   |     |   (0) comments


    Allot AVP of International Pre-Sales, Daniel Keidar, explains how communications service providers can protect infrastructure and service availability from flooding attacks caused by malfunctioning or bot-infected devices connected to their network.
    Upcoming Live Events
    October 18, 2017, Colorado Convention Center - Denver, CO
    November 1, 2017, The Montcalm Marble Arch
    November 1, 2017, The Montcalm Marble Arch
    November 30, 2017, The Westin Times Square
    All Upcoming Live Events
    Infographics
    With the mobile ecosystem becoming increasingly vulnerable to security threats, AdaptiveMobile has laid out some of the key considerations for the wireless community.
    Hot Topics
    Netflix's Lesson in Culture Expectation Settings
    Sarah Thomas, Director, Women in Comms, 6/21/2017
    No Imagination: UK Chip Biz Goes Up for Sale
    Iain Morris, News Editor, 6/22/2017
    Kalanick Steps Down as Uber CEO
    Sarah Thomas, Director, Women in Comms, 6/21/2017
    BT Tech Chief Makes Plea to 5G Chip Vendors
    Ray Le Maistre, International Group Editor, 6/20/2017
    Like Us on Facebook
    Twitter Feed
    BETWEEN THE CEOs - Executive Interviews
    Following a recent board meeting, the New IP Agency (NIA) has a new strategy to help accelerate the adoption of NFV capabilities, explains the Agency's Founder and Secretary, Steve Saunders.
    One of the nice bits of my job (other than the teeny tiny salary, obviously) is that I get to pick and choose who I interview for this slot on the Light Reading home ...
    Animals with Phones
    Live Digital Audio

    Playing it safe can only get you so far. Sometimes the biggest bets have the biggest payouts, and that is true in your career as well. For this radio show, Caroline Chan, general manager of the 5G Infrastructure Division of the Network Platform Group at Intel, will share her own personal story of how she successfully took big bets to build a successful career, as well as offer advice on how you can do the same. We’ll cover everything from how to overcome fear and manage risk, how to be prepared for where technology is going in the future and how to structure your career in a way to ensure you keep progressing. Chan, a seasoned telecom veteran and effective risk taker herself, will also leave plenty of time to answer all your questions live on the air.