KDDI network goes down for two days after maintenance failure

Japan's KDDI was still battling to fully restore mobile services Monday evening, nearly three days after the entire network went down, knocking out banking, transport, payment and other services across the country.

More than 39 million subscribers were affected by the outage, which began early Saturday morning during a routine upgrade.

By 6 p.m. local time Monday, most voice and data services had resumed, but the company could not say when the network would be fully restored.

Minister of Internal Affairs and Communications Yasushi Kaneko declared the failure "a serious incident" under the Telecom Business Law.

KDDI has not given a detailed explanation for the incident, but it appears to be the fault of operator staff rather than an equipment failure.

One version of events is that the outage occurred while the voice switching platform was replaced during regular maintenance. In another version it was a core router that was being switched out.

In any case, the outage had a cascade effect, knocking out both voice and data services and then forcing the company to choke off access as congestion grew.

The disruption affected KDDI's three mobile brands au, UQ Mobile and Povo, and a diverse array of key services.

That included 900 of the Japan Meteorological Agency's 1,300 monitoring stations, 190 Ogaki Kyoritsu Bank ATMs, Japan Rail freight trains, public transport smartcard, on-board calling in Toyota and Suzuki vehicles and emergency calls for help from two hikers in Hokkaido.

The outage also hit rival Rakuten, which has a national roaming agreement with KDDI to supplement its own network. Rakuten said it is still investigating the impact, although the company has said 90% of its data traffic is now carried on its own network.

KDDI CEO Makoto Takahashi apologized at a press conference Sunday morning, acknowledging it was the biggest network outage in the operator's history, Japan Times reported.

He said the company would consider offering compensation to both individuals and companies impacted.

Want to know more about 5G? Check out our dedicated 5G content channel here on Light Reading.

The company's stock dived 4% in early trading before recovering to finish down 1.7%.

It is one of a number of major outages that seem to strike Japanese operators with some regularity.

An NTT Docomo outage last October took out 2 million 3G services for 29 hours.

SoftBank Mobile went down in December 2018, cutting off 30 million subscribers for five hours in an event that it blamed on an Ericsson software error.

KDDI says it's investigating the outage. As a designated "serious incident" – that is, one that affects 30,000 or more users for more than an hour – it will be required to file a report to the government.

It is unlikely to suffer any significant financial penalty, however. Domomo received a mere rebuke for its downtime last year.

But Marc Einstein, chief telecom and digital analyst at ITR Corp, said the latest outage was long-lasting with broad impacts.

"So for that reason I would not be surprised to see more regulation in the sector as a result, similar to what the Japanese government did with the banking system following a series of ATM network outages."

— Robert Clark, contributing editor, special to Light Reading