Singtel's own internet exchange crashed Optus – reports

Did Optus' parent Singapore Telecom crash its network? In its explanation for last week's 14-hour outage, Optus said the failure was caused by "changes to routing information from an international peering network."

Now it's been reported that this third party was in fact Singtel's own internet exchange StiX.

Two sources, including an Optus engineer, pointed to STiX as the source of the crash, Sydney Morning Herald reported Tuesday.

The Optus engineer said the company was unwilling to identify its parent as the source of the disaster because the Singtel board was visiting Sydney last week. Light Reading has sought comment from Singtel.

In its statement, issued Monday afternoon, Optus said it received the routing data following "a routine software upgrade."

"These routing information changes propagated through multiple layers in our network and exceeded preset safety levels on key routers which could not handle these," it said.

"This resulted in those routers disconnecting from the Optus IP Core network to protect themselves."

The nature and scale of the outage meant Optus technicians had to go on-site to physically reboot the system and was also the reason why it took so long to determine the cause of the failure.

Not fit for purpose

The company said it had since made changes to the network to ensure it cannot occur again.

But Mark Gregory, Associate Professor, School of Engineering, RMIT University, said the national outage meant the Optus network "is not fit for purpose."

Writing in The Conversation, he said human error was not unusual, but that it was unusual for a single error to crash an entire network.

"It can be assumed Optus has a number of deficiencies, such as problems with engineering capability, testing, procedures, network redundancy and resilience," he wrote.

The catastrophic failure took down 10 million consumers and 400,000 businesses. Optus now faces a series of inquiries into the event, starting with a senate hearing on Friday.

Optus and rival Telstra will also be brought into the national critical infrastructure regime, requiring the companies to report regularly on their security measures.

It may be cold comfort for Optus, but it’s not the first to be brought undone by a poorly configured router or bad data.

In the longest network outage in recent times, large parts of KDDI's mobile network went down for 61 hours in July 2022 because of a misconfigured network router.

Whatsapp and Facebook went down for six hours in October 2021 after a backbone router configuration went wrong.

In what was possibly the largest of all, Guangdong Telecom's mobile network went off air for five hours in June, affecting tens of millions of customers. The company has never given an explanation but the scale suggests that, as with Optus and KDDI, it was a configuration problem in the core network.