Nine months ago, BT was overly reliant on a particular large language model (LLM) and not hugely impressed with the results. Moving to another brought improvements within just three months and convinced the UK telecom incumbent it must not find itself in the same position again. Rather than pumping resources into an LLM of its own, BT has instead built a platform – with help from AWS – that allows it to switch easily between LLMs depending on its needs. BT calls it the GenAI Gateway.

"Our internal data scientists can solve the various problems using whatever model they want," said Harmeen Mehta, BT's chief digital and innovation officer, at the operator's Adastral Park R&D facility near Ipswich. By matching the "use case" to the most optimal LLM, BT can better manage its costs and generate more accurate results, she told Light Reading. "We don't want to be all in with one."

For the same reason, Mehta, who joined BT from India's Bharti Airtel in March 2021, is equally opposed to the idea of building a model from scratch. "We don't think we need to build our own large language models at all," she said. The stance separates BT from a few other telcos more eager to plunge into the foundational code, but it does not stop BT from fine-tuning models to improve their accuracy and reduce costs. "We have an option as a company to actually say that we will fine-tune the training on our data and not share that back in the public domain," said Mehta. "We have built the entire data fabric that takes care of our privacy layer."

So far, the main impact of generative AI (GenAI), one of the industry's big talking points over the last couple of years, can be seen in the improved "chatbots" deployed for customer care. Aimee, as BT's chatbot is christened, predates the era of ubiquitous GenAI and LLMs, a GenAI building block. But it has been given a "boost" by the technology, said Mehta. "We've pivoted on how we were training Aimee and made her more intelligent so she can answer more queries."

Saving Aimee

Yet Mehta describes Sprinklr, a New-York-based software company, as the organization "powering" Aimee after a deal BT struck with it in March this year. And rather than being a big model builder, Sprinklr seems to have a platform-based approach that resembles BT's. "They started in managing a lot of the social media for customer care," said Mehta. "They provide a platform where a call center agent, the equivalent of a messaging agent, is able to manage multiple customers, interact with them, solve their problems." On its website, Sprinklr now says its platform can integrate with GenAI technologies including Google Cloud's Vertex and OpenAI's GPT models.

While much of the detail remains confidential, the nature of BT's arrangement with Sprinklr and the overarching philosophy that produced the GenAI Gateway appear to have made Aimee less dependent on a specific LLM. As Mehta puts it, the model "becomes a little less important" in the mix. "You don't need the high-end model for every single problem," she said, speaking about BT's broader use of GenAI. "Some problems are just easily as solved with NLP [natural language processing]. If you only want to know the intent of a query, you don't need a large language model to do that."

The approach is "dramatically" changing the financial operations model within BT, Mehta insists. And if chatbots remain the most prominent example of GenAI's deployment, the technology is now spreading through other parts of the organization. Much like mobile rival Vodafone, BT is also using it for summarization of customer calls, especially in the business sector, to identify common complaints or problems. Mehta sees an additional role for the technology in functions where there is what she calls "large document management," such as the legal and bid departments.

Her teams have also made use of Amazon Q, a generative AI-powered assistant for writing various types of documentation, to produce software code now amounting to about a quarter of a million lines. "We're actually working with Amazon to expand what Q can do and what languages Q works in and to get Q to be more accurate as well in its suggestions," said Mehta.

All such examples, though, have generated concern about the impact AI, and especially GenAI, will have on the jobs of today. In May last year, Philip Jansen, BT's former boss, reckoned the operator would be able to reduce its workforce by up to 55,000 jobs before the end of the decade, indicating that AI would claim about 10,000 of them. After cutting 10,000 jobs in its last fiscal year, and with Alison Kirkby as the new CEO, BT left the headline target unchanged when it reported its latest set of annual results. A reduction of that magnitude would leave the operator with just 75,000 employees, including contractors, by 2030, down from about 120,000 in March.

But Mehta thinks AI's main effect will be to refashion skills and roles rather than simply terminate jobs. "What is the new workforce of tomorrow going to look like?" she said. "It's going to have a different shape, and it's going to have a different skillset." The proportions of the workforce in different areas could change, but AI will certainly not end the need for people, she thinks. For the risk-averse, brushing up on those AI skills would seem a sensible idea.

Update: "Aimee" was misspelt "Amy" in the original version of this story and there was a reference to "blockchain" that should have said "document." Both these errors have now been corrected.