Following the recent test of its service provider virtualization and cloud portfolio by independent test lab EANTC, Cisco is ready for further third-party assessments to help show the market, and the rest of Cisco, how far the vendor has come in terms of its carrier SDN and cloud developments.
In February this year, a team of engineers from European Advanced Networking Test Center AG (EANTC) spent two weeks at Cisco Systems Inc. (Nasdaq: CSCO)'s San Jose facilities, putting a number of the vendor's virtualization and cloud systems and applications through their paces, with EANTC effectively acting as a prospective service provider customer. The EANTC team then delivered an in-depth and comprehensive report of its findings to Light Reading, which funded the test process. (See Validating Cisco's Service Provider Virtualization & Cloud Portfolio and Cisco Lab Test Validates New IP World.)
There was a degree of apprehension at Cisco about an external team being let loose on the company's virtualization and cloud systems: Testing hardware platform capabilities is well understood by all parties, but verifying the capabilities of virtualization systems had not been done before. In addition, as Cisco's senior director of Products & Solution Marketing, Sanjeev Mervana, notes, this was the first time that the vendor's WAN Automation Engine (WAE) and Network Service Orchestrator (NSO) had been put through a comprehensive test by an external, independent team.
"The reaction from the engineering teams was quite positive and supportive when the concept of having an independent test house come in and validate the capabilities of various solutions," was discussed, Mervana tells Light Reading. "However, there was a bit of anxiety and a 'fear of the unknown' when it was disclosed that the test results would be made public" by Light Reading, he adds, as there were concerns about what might happen if anything went wrong during the tests, especially as EANTC's methodology was unknown prior to the tests. "Nobody likes to expose any shortcomings in public," adds Mervana.
Ultimately, though, "the teams stepped up to the challenge and were very excited" about the potential outcome of the planned validation and verification processes, as they wanted to "finally put to rest some FUD [fear, uncertainty and doubt] in the market about Cisco not leading in these areas," adds Mervana with a marketing flourish.
So what did the Cisco teams make of EANTC's processes and test methodology? Was EANTC's approach of acting like a service provider that wants to put Cisco's systems through near real-world scenarios surprising? "I would not call it a surprise, but the EANTC team were quite thorough and very professional in their questions and methodology as they were portraying themselves as our customers -- and our customers are really not shy in demanding things from us," adds the Cisco man playfully. "Some of the test methodology certainly raised the eyebrows of our test engineers, especially when the tests focused on the aspects of high availability and service restoration," adds Mervana. (See pages 14 and 15 of the report.)
"What was really amazing was the level of detail [applied by] the EANTC team … around the entire journey of transforming an existing business VPN service to a virtualized business service in a brownfield environment. By acting as a service provider, the EANTC team went through various stages and were able to test real scenarios that customers spend a lot of time, resources and money going through today," notes the Cisco man. "These tests and its results will help save many service providers a lot of time and resources in the evaluation of various solutions out there in the market today -- they can help them decipher what's real and what's not," he adds.
The tests results have also been useful for Cisco internally. "The reaction has been very positive within broader Cisco teams, as many [other internal teams] did not realize how far Cisco has come in this journey of SDN and NFV -- these tests acted as solid proof points," states Mervana, who adds that Cisco is ready for further external verification of its systems. "We always welcome such independent tests."
And there's no doubt that such tests are valuable for the market in general, so Light Reading will be brokering and funding further SDN, NFV and cloud independent verification processes in the coming months and publishing the results.
— Ray Le Maistre, , Editor-in-Chief, Light Reading