BT Threatens to Ditch OpenStack

The six challenges
Providing more detail on the six challenges he outlined, Willis says the first issue is connecting VNFs to the infrastructure. OpenStack does this in a sequential manner, with the sequence serially numbered in the VNF, but the difficulty comes when trying to verify that the LAN has been connected to the correct LAN port, the WAN has been connected to the correct WAN port and so on. "If we get this wrong for a firewall function it could be the end of a CIO's career," says Willis.

OpenStack also appears to come up short in the area of service chain modification, whereby new services are inserted into the mix. "Let's say we deliver a router and a firewall to the customer and they're connected together and then the customer asks for a WAN accelerator," explains Willis. "There is no way to do that in OpenStack -- you have to disconnect the interface and reconnect."

BT has tested the disconnection and reconnection of interfaces with a range of VNFs from different vendors and had mixed results. In the worst-case scenario, VNFs have locked up entirely and the operator has had to remove them and start from scratch.

One answer to the second challenge might be deleting the service chain and starting afresh, according to Willis, but this could have repercussions for customer service.

Lack of scalability is a further problem. The BT executive reckons a single OpenStack controller can be expected to manage about 500 computing nodes. "In our virtual enterprise CPE scenario we're talking of the order of 100,000 CPEs to manage and so that scalability figure is not very good."

The "start-up storms" to which he refers can happen when, say, a fiber connection is broken and then subsequently fixed. "Imagine we have a controller with 100,000 nodes and all these distributed agents try to reconnect at the same time," says Willis. "They're all using encryption and that's slow and computationally intensive and the only way to get the network back is to phone up customers and tell them to turn their nodes off and on again one at a time -- that is not going to be very pleasant with 100,000 customers."

Securing OpenStack over the Internet is an additional headache for Willis. During his own tests, he took a computing node and put this on the end of a DSL line connected to the Internet, but to make it work he had to open more than 500 pinholes in his firewall to the controller. "That's like a highway -- it's a serious problem," he says. "We definitely need to consider how we secure OpenStack over the Internet because we will be having virtual enterprise CPEs on the end of Internet connections."

Finally, there is a need for backwards compatibility between the versions of OpenStack, says Willis, who rules out any possibility that BT will be able to run the whole network on the same version.

"It's not feasible for managing a carrier service because I've got thousands of customers and each customer may have a different planned engineering works," he says. "We have to have backwards compatibility and be able to run multiple different versions on the computing node from a single controller."

— Iain Morris, , News Editor, Light Reading