Karsten Nohl has the unusual job of hacking into companies at their behest. Known as "red teaming," the process is the IT world's equivalent of driving crash test dummies into brick walls or firing dead chickens into a grounded plane's jet engines.

The basic idea is to expose the vulnerabilities safely (this, remember, is how Nohl makes a living) and then patch them up before there is any real-world damage. But what Nohl and Security Research Labs (SRL), his Berlin-based firm, found in the 5G networks they examined was "very scary," he told MCH2022, a hacker camp in the Netherlands this week.

On aggregate, the handful of 5G networks SRL was invited to hack were almost boringly easy to penetrate. In one case, an isolated website created by a company developer provided a point of entry to IT systems.

While it might have seemed like a dead end, that website also ran in Docker, a software-management tool used across operations. Thanks to weaknesses in the Docker setup, SRL was able to "break out" of it and infiltrate Kubernetes, an underlying platform responsible for managing cloud applications.

From there, it was not long before the good-guy hackers were rummaging through customer information like a burglar through a jewellery drawer.

Figure 1: Karsten Nohl explain why open RAN is so vulnerable to hacking.
(Source: YouTube)

Old security threats have been amplified. Phishing, whereby hackers trick people into revealing sensitive data, would previously have targeted a few system administrators.

But the adoption of Docker and Kubernetes, both open-source platforms, means hundreds of people across various companies might be writing code for the configuration of a mobile network.

"If you can phish any one of them, there is a good chance you can adversely affect the mobile network," said Nohl.

Essentially, this community-based approach has multiplied the opportunities for the criminally minded. If any part of the chain is hacked, the mobile network is at risk, says Nohl, questioning some of the familiar, no-security-in-obscurity claims about open-source code.

"I'm not so sure this many-eyes-make-bugs-go-away argument applies to software that is used in only one or two companies," he said.

"You post it on the Internet and people don't start looking for bugs in it. The hacker will use that information. Some level of obscurity sometimes helps in protecting APIs [application programming interfaces]."

The rush to cloudify

In Nohl's assessment, much of the blame for this mess lies not with anything telco-, network- or 5G-specific but with a rush to virtualize and cloudify operations.

Theoretically, operators should be able to segregate resources more easily with virtualization to mitigate risk. But this rarely happens, according to Nohl.

"Most deployments do not make use of this fine-grained configurability," he said. When different components are sharing hardware and one overloads the system, the others can suffer, too.

"If you configure badly, it is a definite loss."

When SRL infiltrated one company's systems through that isolated developer website, it took advantage of a configuration that assigned "privileged" and "system admin" rights to the Docker container hosting the website (a container is basically a software package).

With these, a hacker can access the Linux kernel of the host machine – the very heart of the operating system – in "less constrained ways than [via] a non-privileged container," said Nohl. Settings assigned to an unimportant website were the equivalent of the key under the plant pot outside the supposedly secure home.

The broad theme of Nohl's presentation was that open RAN has made 5G hacking "a lot more interesting."

Strictly speaking, of course, open RAN is about the interfaces that link different parts of the RAN, but it is often conflated with virtualization or cloudification – purely because they are integral to most open RAN deployments.

When Europe's five largest operators – Deutsche Telekom, Orange, Telefónica, Telecom Italia and Vodafone – last year issued a list of open RAN technical priorities, Kubernetes was identified as the "mainstream implementation" of the cloud platform hosting open RAN functions and applications.

Want to know more about 5G? Check out our dedicated 5G content channel here on Light Reading.

What's debatable is Nohl's view that a future, cloudified 5G network would feature "several hundred" data centers in a country the size of Germany.

Nohl seems to believe this many would be needed to support tomorrow's low-latency applications. Others are not so sure.

But if that is how 5G evolves, then each of those data centers would feature Kubernetes clouds, and each of those Kubernetes clouds would support dozens of Docker containers. The sheer numbers involved mean everything must be highly automated.

"Instead of having people lock into boxes once a year and changing a little bit, now you have scripts running all the time that try to optimize the network, that keep reconfiguring everything," said Nohl.

"It is living organism-style network that is self-optimizing. To me, that is very scary – knowing that it continuously changes. How are you going to test it and say for the next few years I know it's secure? You test it and even five minutes later it has a different state."

"The silver lining is that at least we remove the human error source from operations," he continued.

"If everything is software nobody can fat-finger or get phished anymore. But, of course, we introduce another, maybe more serious, human error source in that many developers are now involved who – through some magical CI/CD [continuous integration/continuous delivery] pipelines – create and push down these scripts."

Once inside one company's systems, SRL stumbled on something called the RAN Intelligent Controller (RIC), a trumpeted software feature of open RAN that allows telcos to optimize the network in real time. Poor configuration of a Docker container allowed Nohl's team to break out of that and access a RIC software component running on numerous Kubernetes clouds.

"See how few steps are required to take control of an entire network," said Nohl.

The coming 5G apocalypse

Relatively few networks are built this way now. Omdia, a sister company to Light Reading, believes open, virtualized RAN will account for about 5% of spending on RAN products this year, rising to 15% in 2026.

That could provide a clue about the identities of the telcos Nohl hacked, unless the red teaming was aimed at trial deployments only (Nohl was not up for naming clients, understandably).

Open RAN's cheerleaders like to refute this type of criticism as mendacious negativity originating with the big kit vendors that open RAN could hurt.

But Nohl is an awkward and troubling critic. He is very highly regarded as a cryptography expert and hacker, for one thing, with an impressive résumé.

Among other things, he previously worked on an interim basis as the chief information security officer for Jio, India's biggest operator, and Axiata, a Malaysian telecom conglomerate. As a man entrenched in software, he has no apparent vested interest in sticking up for Chinese or Nordic kit vendors selling proprietary boxes.

The broad fix, in Nohl's opinion, seems to lie in paying more attention to security and making "good use" of configuration settings. Security patching needs to be done on a regular basis, he said.

But none of this is happening by default in telecom, based on his observations. And if the industry is as dead set on cloudification as it appears, there is a lot that could go wrong.

Related posts:

— Iain Morris, International Editor, Light Reading