Vodafone performs brain surgery on chatbot, ditching Watson and LUISVodafone performs brain surgery on chatbot, ditching Watson and LUIS

The updated SuperTOBi is based on OpenAI, meaning the older IBM and Microsoft technologies used with the original TOBi will be phased out.

Iain Morris, International Editor

July 5, 2024

5 Min Read
Superman flying over city at night
Is GenAI SuperTOBi's Kryptonite?(Source: Pictorial Press Ltd/Alamy Stock Photo)

TOBi, the customer-facing chatbot Vodafone launched a few years ago, has emerged from the phone booth wearing a giant red S. Minus the typical spandex, it is now SuperTOBi, with the power to fly through queries and zap problems that were too much for its puny, Clark Kent-like former self. "Is it a customer service agent?" whisper amazed listeners, straining to pick out any machine-like gibberish. "No, it's SuperTOBi," answer Vodafone's actual customer service agents, polishing nails while SuperTOBi does the work.

Old TOBi was conceived long before Sam Altman had become a household name and generative artificial intelligence (henceforth GenAI) a topic of pub conversation. Next to OpenAI, now rapidly colonizing the planet, it apparently looks Neanderthal. "We have chatbots interacting in the market across our footprint, and these chatbots are with an AI that is more rudimentary, and three or four years' old, with a baby AI-type brain," said Ignacio Garcia, Vodafone's global data and analytics director, at a press roundtable in London.

Vodafone had relied on two older machine-learning platforms to create TOBi – the well-known IBM Watson, famous for winning US quiz show Jeopardy in 2011, and the more obscure Microsoft LUIS. (Microsoft says the acronym stands for "Language Understanding," but this would surely make it LU? Then again, perhaps this basic language inability reflects the system's shortcomings).

In the AI version of a brain transplant, Vodafone is now replacing these two platforms with GenAI technology. "We have some markets with IBM Watson and some markets with Microsoft LUIS and, as we scale up across all our markets, we'll replace that with our GenAI capabilities," said Scott Petty, Vodafone's chief technology officer. The identity of the new, more advanced brain was undisclosed at the roundtable. But it was outed in a subsequent press release as OpenAI, the Microsoft-funded Sam Altman vehicle, running on Azure, Microsoft's cloud.

Learning curve

At first, switching Vodafone's TOBi trainers from Watson and LUIS to OpenAI was like asking a team of dog handlers to work with apes. Results were disappointing, with answers to questions posed by staff correct only 75% to 78% of the time, reports Garcia. "Why is that? Our prompting was not perfect, and we were trying to force through our own way of saying things, because that is how we were using the previous AI, rather than using the power of the language model." Subsequent changes to prompting boosted the success rate to a "super high" 90%, he said.

SuperTOBi was initially exposed to just a few thousand customers, but it is now being used by a million, said Garcia, and appears far more popular than the original chatbot. "We are getting 50% more satisfaction, and 50 is a big number from the customers." Less constrained by menus and the challenges of navigating them, customers can more freely ask natural language questions. 

The supercharged chatbot has now been trained to understand 11 different languages, but Vodafone has minimized the effort involved through a kind of central repository dubbed AI Booster (used for AI applications besides SuperTOBi). Using some open-source tools, this also guards against bad behavior and inappropriate answers by curating the data Vodafone holds and updates. Even so, Vodafone has not been able to avoid the need for linguistic experts across so many different languages to do effective prompting.

How long, then, before SuperTOBi is speeding to the rescue of distressed customers across the whole footprint? In its official release, Vodafone says it spending €140 million (US$152 million) on a "transformation of the customer experience" this fiscal year. It has already unleashed SuperTOBi in Italy and Portugal and will do the same in Germany and Turkey later this month. Other markets are to follow this year, it says.

Latency challenge

Deployment, though, is not straightforward, partly because Vodafone's networks must be able to cope. "First of all, we launched text-based chat," said Petty. "TOBi does voice, and voice is much, much harder because latency is much more important. In chat, you're quite happy to wait three or four seconds for your answer. If you're talking to a chatbot and it doesn't answer in less than a second, you are probably going to hang up on that call and go back to a contact center agent."

The standalone (SA) version of 5G, which combines a new core with the radio technology already rolled out, could make a big difference to latency, Petty reckons. But it has not been widely deployed so far across Europe. In the UK, where it is headquartered, Vodafone has promised to invest £11 billion ($14.1 billion) in building a nationwide SA network if authorities approve its proposed merger with Three, and the new Labour government has come under immediate pressure from Three's boss.

"The UK's connectivity is way behind where it needs to be and we look forward to working with them to address this," said Robert Finnegan in a prepared statement this morning, hours after Labour had swept to power with a huge majority. "The merger of Vodafone and Three enables us to invest £11 billion in a dedicated 5G network that would give a huge boost to companies and individuals across the country."

Petty also continues to worry about the accuracy of the answers SuperTOBi generates. A less cautious telco could probably roll it out in six weeks to three months, but this would run the risk of a "negative customer experience," he said. Petty, accordingly, thinks it could be up to a year before SuperTOBi is everywhere.

Fans of superhero stories know the characters have not always been a hit with fictional law enforcement agencies, left jobless by their brilliance. Is SuperTOBi similarly bad news for the people working as customer service agents today? Despite widespread fears about the impact of GenAI on jobs, Petty insists people won't really be left polishing their nails as they wait to be laid off. "While TOBi is great at managing interactions, and millions of interactions, we can then use our contact center agents to do much higher-value work," he said. Staff must hope the Vodafone bosses in charge of budgets have the same view.

Read more about:

AIHeadcountEurope

About the Author

Iain Morris

International Editor, Light Reading

Iain Morris joined Light Reading as News Editor at the start of 2015 -- and we mean, right at the start. His friends and family were still singing Auld Lang Syne as Iain started sourcing New Year's Eve UK mobile network congestion statistics. Prior to boosting Light Reading's UK-based editorial team numbers (he is based in London, south of the river), Iain was a successful freelance writer and editor who had been covering the telecoms sector for the past 15 years. His work has appeared in publications including The Economist (classy!) and The Observer, besides a variety of trade and business journals. He was previously the lead telecoms analyst for the Economist Intelligence Unit, and before that worked as a features editor at Telecommunications magazine. Iain started out in telecoms as an editor at consulting and market-research company Analysys (now Analysys Mason).

Subscribe and receive the latest news from the industry.
Join 62,000+ members. Yes it's completely free.

You May Also Like