Operators face tricky decisions as they try to get virtualized networks off the ground.

Iain Morris, International Editor

September 30, 2019

9 Min Read
The Eye-Watering Cost of Multivendor Networks

"Multivendor" almost sounds like molto bene, the Italian for "very well," when spoken by Enrique Blanco, the Spanish chief technology officer of Telefónica. But there is little to commend the status of virtualized telecom networks built by more than one supplier. Virtualization was supposed to be a bright spark that reinvigorates the telecom sector. Seven years after conception, it has become a juvenile delinquent that persistently offends.

The main problem is when operators are trying to build networks with multiple suppliers. Even if the operator sticks with a single vendor, an operator can expect its operating costs to rise with the introduction of virtualization before they eventually drop to about 91% of the original amount, according to China's Huawei. With a multivendor rollout, operating costs end up at 146% of the previous figure, it says.

A shorter "time to market" (TTM) is usually seen as the other direct benefit of virtualization. But even here there may be limited improvements if an operator is determined to use numerous suppliers. Average TTM ends up around six times greater on a multivendor than a single-supplier project, according to Huawei. That is a massive difference for telecom executives who believe their survival depends partly on how quickly they can move.

Figure 1: Bill Shock Telefonica's Blanco is determined to go multi-vendor despite concern about the expense. Telefónica's Blanco is determined to go multi-vendor despite concern about the expense.

This is a curious analysis by Huawei. On the one hand, its natural desire to be the only supplier in the mix gives it an incentive to paint multivendor projects in a bad light. On the other, Huawei needs to show willingness to operators that refuse to be knocked off the multivendor path.

Yet Telefónica's Blanco agrees that a multivendor rollout could be costly. "It is easier to do with a single vendor in a single core and it will be cheaper," he told attendees at this month's 5G Core Summit in Madrid.

Despite his reservations, Blanco has blackballed single-vendor entries as Telefónica invites interest from suppliers of 5G core network technologies. The main reason is that he does not want Telefónica to be overly reliant on one provider. Forthcoming 5G services could play a vital role in society, he said during a presentation in Madrid. Trusting everything to one supplier would hardly be wise.

The interest in multivendor networks is nothing new, of course. When virtualization first appeared, a big selling point was that it would supposedly give operators more freedom to combine software and hardware companies in their networks. The basic premise is to end the tight integration of hardware and software in telecom equipment. Ideally, an operator should be able to run any software vendor's code on common, off-the-shelf servers. Thanks to this "disaggregation," moving between different suppliers would become much easier, telcos hoped.

Want to know more about 5G? Check out our dedicated 5G content channel here on
Light Reading.

Recent developments are persuading executives that multivendor networks are a necessity. Years of consolidation, including Nokia's 2016 takeover of Alcatel-Lucent, have concentrated network power in the hands of a few giant vendors. None looks in the best of health. After retreating from some markets to focus on mobile, Ericsson now faces a $1 billion US fine for corruption and the risk of non-US probes into its past activities. Nokia is still racking up losses, according to International Financial Reporting Standards, and is monitoring its Chinese business amid signs of growing support for local vendors. Huawei is banned from acquiring US components and blocked from doing business with some Western operators.

But working with smaller companies has turned out to be no easy thing, forcing operators to do more work as they stitch a system together. Making matters worse is the push toward open source technology and microservices, lightweight alternatives to the somewhat clunky virtual machines that appeared in the first wave of virtualization. "There are more pieces you need to bring together," said Franz Seiser, the vice president of core network development for Germany's Deutsche Telekom, at the Madrid show. "The whole philosophy has not been designed to make it easier to run the system. The opposite is the case. That is the price you pay."

This integration headache partly explains why costs may rise. Some operators are turning the job over to professional systems integrators such as India's Tech Mahindra, which is notably supporting ecommerce giant Rakuten with the deployment of a new virtualized and multivendor mobile network in Japan. But the use of these specialists means there is another bill to pay. Making do without them puts more pressure on the in-house operations team. And when something goes wrong in a multivendor network, identifying the responsible party might not be straightforward. There is no longer just "one throat to choke," say operators.

Next page: The benefits and risks of monogamy

The benefits and risks of monogamy
Given the apparent hassle, some telcos are moving ahead with single-vendor deployments. In Belarus, Velcom claims to have built a fully virtualized mobile core with ZTE, a Chinese equipment vendor that ran into problems last year when it was temporarily banned from acquiring US components. "There are benefits and risks, but we agreed to do this," said Valery Barai, Velcom's head of core and service network planning, at the Madrid event.

Belarusian authorities were evidently less worried than some other watchdogs about overreliance on Chinese equipment. Nor did that appear to bother Velcom, even though that operator is a part of the Telekom Austria Group. On the regulatory front, the main concern was about lawful interception, or the Belarusian government's ability to access phone records and other data held by Velcom. Belarusian laws resemble those in the old USSR rather than Europe's system, said Barai. "There are lots of requirements and it wasn't easy to cover everything."

From a technical and business perspective, Velcom has tried to mitigate the risks in a couple of ways. From the outset, it insisted on "no customization" and a fully standards-based approach. It also made sure its own research-and-development team was heavily involved in the project so that any deviation from 3GPP standards was understood. Accordingly, a team of 30 engineers from the vendor worked alongside as many as 80 experts from Velcom. The entire project was finished in a year and the only core network function that has not been virtualized is Velcom's media gateway, said Barai. Even so, the system is not what the industry would describe as cloud-native, meaning it is not based on a microservices architecture. Migrating to that is one of the main upcoming challenges.

Despite telco anxiety, not everyone thinks multivendor networks necessarily mean additional expense in the long run. Croatia's Hrvatski Telekom, majority owned by Germany's Deutsche Telekom, does not have a single-vendor strategy and claims to have realized major cost savings through virtualization so far. The problem, according to Goran Car, its operating director of core network and service platforms, is that some operators are not prepared for the sweeping changes this approach demands. "Not everyone wants to change or is ready for change and you need to push it," he said in Madrid.

For Hrvatski Telekom, change has involved taking a more active role in technology development. Noting a lack of alternatives to the likes of Ericsson and Nokia, the Croatian operator has worked with specialists to develop the software it needs for its virtualized network. Bigger players, including Deutsche Telekom and France's Orange, have similar ambitions. In 2017, Orange and some venture capital partners promised to invest up to €100 million ($110 million) in network start-ups over the subsequent three to four years. Deutsche Telekom announced the same commitment a few months later in partnership with the Telecom Infra Project (TIP), a Facebook-led initiative aimed at spurring innovation and lowering costs in network equipment markets.

For more NFV-related coverage and insights, check out our dedicated NFV content channel here on Light Reading.

Yet despite all the talk, few network start-ups have made an impact so far, and most operators remain hitched to their traditional suppliers. In June, Mansoor Hanif, the chief technology officer of UK regulator Ofcom, and a former executive at UK mobile operator EE (now owned by BT), noted "skepticism" about the likelihood mobile operators would buy products from start-ups, including those backed by TIP. A new Ofcom plan to sell spectrum to companies that are not telcos is partly aimed at providing an alternative market for new vendors.

Nearly everyone insists that culture, and not technology, is the big problem. Telco employees are not used to handling software and even less familiar with the working practices of a typical software firm. Their technical staff think Python is a non-venomous snake and still use acronyms that became unfashionable at the same time as permed hair. Their commercial models are misaligned. Their sales and marketing departments understand "aaS" as something you sit on.

Changing all this could prove extremely costly. Whether operators try to retrain existing members of staff or introduce new talent into the workforce, the process could also take years. And time is certainly not on their side. A tsunami of data traffic on telecom networks has not brought a surge in revenues with it. The Internet companies riding that wave are a growing threat. "There are some very software-centric companies out there and if we want to be competitive we need to come up with things much faster than in the past," said Deutsche Telekom's Seiser. If they cannot, telcos may pay a much heavier price than the cost of any transformation.

Related posts:

— Iain Morris, International Editor, Light Reading

Read more about:

EuropeAsia

About the Author(s)

Iain Morris

International Editor, Light Reading

Iain Morris joined Light Reading as News Editor at the start of 2015 -- and we mean, right at the start. His friends and family were still singing Auld Lang Syne as Iain started sourcing New Year's Eve UK mobile network congestion statistics. Prior to boosting Light Reading's UK-based editorial team numbers (he is based in London, south of the river), Iain was a successful freelance writer and editor who had been covering the telecoms sector for the past 15 years. His work has appeared in publications including The Economist (classy!) and The Observer, besides a variety of trade and business journals. He was previously the lead telecoms analyst for the Economist Intelligence Unit, and before that worked as a features editor at Telecommunications magazine. Iain started out in telecoms as an editor at consulting and market-research company Analysys (now Analysys Mason).

Subscribe and receive the latest news from the industry.
Join 62,000+ members. Yes it's completely free.

You May Also Like