Orange CTO: Now's not the time to invest in Nvidia chips for AI

Bruno Zerbib is not your traditional Orange board member. His colleagues past and present have often spent their whole careers at the French telecom incumbent, even joining it immediately after they attended one of the grandes écoles, France's higher-education Ligue 1. The man now wearing the mantle of chief technology and innovation officer had not been employed by Orange in a significant role before June last year. Zerbib's resume shows he has spent most of this century living and working in California for some of the world's best-known Silicon Valley firms. His accent carries undertones of America.

This atypical profile partly explains his appointment, according to one company source. US Internet giants and the technologies they have unleashed are quickly reshaping Orange and other telcos, just as those same telcos previously reshaped economies through mobile and data connectivity. A CTIO with experience on the other side of the fence – both geographically and industrially – may look invaluable as outside technological forces including generative artificial intelligence (GenAI), accelerated computing and the public cloud pose unanswered questions for telcos.

It is AI in its various forms that has, unsurprisingly, become one of the big priorities for Zerbib amid a lack of telecom-industry consensus on the right next steps. Germany's Deutsche Telekom, the biggest service provider in Europe, thinks operators should build their own large language models (LLMs). Vodafone does not. Japan's Softbank is investing in Nvidia's graphical processing units (GPUs), the versatile chips used in hyperscaler facilities for LLM training. Many are wary. The phenomenon of "inferencing," the rollout of AI applications once LLMs are fully trained, could bring revenue opportunities for telcos at the network "edge," some hope. Others with an eye on recent history are skeptical.

The camp of the Nvidia skeptics

When it comes to Nvidia, Zerbib clearly belongs in the camp of the skeptics. "This is a very weird moment in time where power is very expensive, natural resources are scarce and GPUs are extremely expensive," he told Light Reading during an interview at this year's Mobile World Congress, held last month in Barcelona. "You have to be very careful with our investment because you might buy a GPU product from a famous company right now that has a monopolistic position." Low-cost alternatives are coming, said Zerbib.

He is not the only telco individual right now who sounds worried about this Nvidia monopoly. The giant US chipmaker's share price has doubled since mid-October, valuing Nvidia at nearly $2.2 trillion on the Nasdaq. But a 6% drop on March 8, sparked by wider economic concerns, illustrates its volatility. Its gross margin, meanwhile, has rocketed by 10 percentage points in just two years. The 77% figure it reported last year screams monopoly. And Nvidia controls not just the GPUs but also the Infiniband technology that connects clusters of them in data centers. Howard Watson, who does Zerbib's job at the UK's BT, wants to see Ethernet grow muscle as an Infiniband alternative.

But Zerbib spies the arrival of alternatives to GPUs "for a fraction of the price, with much better performance" in the next couple of years. In his mind, that would make an Orange investment in GPUs now a very bad bet. "If you invest the money at the wrong time, depreciation might be colossal and the write-off could be huge," he said.

What, then, could be the source of these GPU rivals? Established chipmakers and chip design companies such as Intel and Arm have pitched other platforms for AI inferencing. But disruption is likely to come from the hyperscalers or the startups they eventually buy, according to Zerbib. "Right now, the hyperscalers are spending billions on this," he said. "There are startups working on alternatives, waiting to be acquired by hyperscalers, and then you are going to have layers of optimization."

While Zerbib did not throw out any names, Google's own tensor processing units (TPUs) have already been positioned as a GPU substitute for some AI workloads. More intriguing is a startup called Groq, founded in 2016 by Jonathan Ross, an erstwhile Google executive who led the development of those TPUs. Tareq Amin, the CEO of Aramco Digital and a former telco executive (at Japan's Rakuten), thinks Groq's language processing units (LPUs) might be more efficient than GPUs for AI at the edge.

Efficiency is paramount for Zerbib as Orange strives to reach a "net zero" target by 2040. "We need to reduce by ten the amount of resource consumption for a given number of parameters," he said. "It is unacceptable. Right now, we are going to hit a wall. We cannot talk about sustainability and then generate a three-minute video that is equivalent to someone taking a car and doing a long trip. It makes no sense."

Models of all sizes

Today, he says he prefers to invest in people rather than GPUs. For Orange so far that has meant equipping about 27,000 members of staff (Orange had 137,000 employees worldwide at the end of last year) with AI skills. Unlike certain other telcos, however, Orange appears to have no interest in building an LLM from scratch. Instead, Zerbib is an advocate of fine-tuning for telecom purposes the LLMs that already exist.

Orange has, accordingly, spent money on building an AI tool it can effectively plug into the most appropriate LLM for any given scenario. "We have built internally a tool that is an abstraction layer on top of all those LLMs and based on the use case is going to route to the right LLM technology," said Zerbib. "We are very much proponents of open models, and we believe that we need to take huge models that have been trained at scale by people that have invested billions in infrastructure."

Available like items of clothing in numerous different sizes, future language models – including those adapted by Orange – are likely to be worn at multiple types of facilities, he clearly believes. Zerbib breaks them down into three categories: small language models, featuring up to 10 billion parameters and deployable on user devices; medium-sized models, which include up to 100 billion parameters and can be hosted at the network edge; and the very largest, trillion-parameter LLMs that will continue to reside in the cloud.

With GenAI, an employee could feasibly communicate "in layman's terms" with a telco's NOCs and SOCs (its network and service operations centers) to find out "what is going on," said Zerbib. Language models running at the network edge might also support applications for customers. "Let's say you have a French administration that needs to offer certain services to constituents throughout the country, and they want to make sure they do that with GenAI and have a distributed architecture to do it everywhere," said Zerbib. "Well, this is an opportunity for a telco to say we might provide an infrastructure to run those distributed LLMs that will handle social administration."

The most obvious AI attractions are still on the cost side. AI-generated insights should clearly help telcos reduce customer churn, minimize network outages and "essentially extract more value from the infrastructure for the same amount of capital investment," as Zerbib describes it. The telco dream, though, is of AI as a catalyst for sales growth. "There is a question mark with slicing, with the emergence of new use cases and low-latency capabilities," said Zerbib. "We might be able to monetize that." Conscious of the telecom sector's historical tendency to oversell, Zerbib is – for now – putting a heavy emphasis on that "might."