Sponsored By

Google: 'Great' Data Center Networks EssentialGoogle: 'Great' Data Center Networks Essential

With Moore's Law coming to an end and I/O at a bottleneck, future computing growth will be driven by networking power, says a top Google engineer.

Mitch Wagner

June 18, 2015

4 Min Read
Google: 'Great' Data Center Networks Essential

SANTA CLARA, Calif. -- Open Networking Summit -- Delivery of computing power will depend on networking in the future, Amin Vahdat, fellow and technical lead for networking at Google (Nasdaq: GOOG), said at a keynote here Wednesday.

Compute platforms face coming bottlenecks that will choke performance, he says. The end of Moore's Law is here, or will be here in a few years, requiring change for computing and distributed computing. Storage capacity has increased hugely but I/O isn't keeping up.

That leaves networking as the driver of future growth.

"Networking is at an inflection point in driving next-generation computer architecture," Vahdat said. "What computing means will largely be dependent on our ability to build great networks over the years."

Figure 1: Google Man Google takes advantage of the control it exerts inside its data centers to drive network efficiencies, says Amin Vahdat, a top engineer for the cloud company. Google takes advantage of the control it exerts inside its data centers to drive network efficiencies, says Amin Vahdat, a top engineer for the cloud company.

Data center networking, in particular, is a special requirement for Google, Vahdat said. Google's services depend on its data centers.

"We build building-scale -- warehouse-scale -- computers. Just row after row of computers and row after row of storage," he said. "And you can't build a world-class distributed computing infrastructure without world-class networking infrastructure."

Google takes advantage of the control it can exert within its own data centers to drive efficiency and scalability, Vahdat said.

Last year at this conference, Vahdat described its Andromeda SDN infrastructure used to link data centers worldwide. (See Google, Microsoft Challenge Service Providers.)

Andromeda slices the physical network into VPNs for customers. It's complemented by Freedome, Google's campus-level interconnect network, and Jupiter, within the data center.

Data center networking has different advantages and problems when compared with sending traffic over the public Internet and wide-area networks. The chief distinction is that a data center is completely under the control of a single organization, unlike the "capital-I Internet," Vahdat said.

"You build it and you control it," he said.

Other distinctions: round-trip times are "tiny," the data center is "massively multipath," bandwidth is more plentiful and uniform. The network makes little use of buffering, and bandwidth needs are massive -- the aggregate bandwidth of the "entire Internet" is "in one building," Vahdat said.

When Google built out its data centers, it had special needs that could not be met with off-the-shelf equipment.

"We could not buy for any price a data center network that would meet the needs of our deployments," Vahdat said. Products were focused on individual boxes, with each box controlled by its own command line interface, making management an overwhelming problem. "The notion of managing a thousand boxes as if they were a thousand boxes didn't cut it."

Google leveraged three key principles: Using merchant silicon in Clos topologies with centralized control.

Google needs to keep up with explosive growth. Over six years, data center bandwidth needs have grown fifty-fold, outpacing Moore's Law. A typical network today might have 10,000 or more switches, with a quarter million links and 10 million routing rules, Vahdat said.

Find out more about key developments related to the systems and technologies deployed in data centers on Light Reading's data center infrastructure channel

Google implemented several generations of data center networking technology to meet its needs. The first generation, code-named Firehose 1.0, didn't see production. Google tried to integrate networking with servers, which didn't work out, Vahdat said.

Firehose 1.1 went live, using the same technology and traditional switches. Initially, Firehose 1.1 was deployed as a "bag-on-the-side," with servers connected both to previous generation cluster routers and the new generation. When Google became confident in the new network, it eliminated the old, Vahdat said.

Firehose was followed by Watchtower, then Pluto running at 10 Gbit/s; Saturn, the fourth generation, has 288 ports of 10 Gbit/s making up the edge aggregation block.

Finally, Jupiter leverages the latest generation of merchant silicon, with 80 Tbit/s bandwidth, hosting an SDN software stack using OpenFlow.

Facebook also relies on efficient data centers to drive its services, and has invested heavily in pushing open source hardware and software design. (See Facebook Reinvents Data Center Networking.)

More from the Open Networking Summit:

— Mitch Wagner, Circle me on Google+ Follow me on TwitterVisit my LinkedIn profileFollow me on Facebook, West Coast Bureau Chief, Light Reading. Got a tip about SDN or NFV? Send it to [email protected].

About the Author(s)

Mitch Wagner

Executive Editor, Light Reading

San Diego-based Mitch Wagner is many things. As well as being "our guy" on the West Coast (of the US, not Scotland, or anywhere else with indifferent meteorological conditions), he's a husband (to his wife), dissatisfied Democrat, American (so he could be President some day), nonobservant Jew, and science fiction fan. Not necessarily in that order.

He's also one half of a special duo, along with Minnie, who is the co-habitor of the West Coast Bureau and Light Reading's primary chewer of sticks, though she is not the only one on the team who regularly munches on bark.

Wagner, whose previous positions include Editor-in-Chief at Internet Evolution and Executive Editor at InformationWeek, will be responsible for tracking and reporting on developments in Silicon Valley and other US West Coast hotspots of communications technology innovation.

Beats: Software-defined networking (SDN), network functions virtualization (NFV), IP networking, and colored foods (such as 'green rice').

Subscribe and receive the latest news from the industry.
Join 62,000+ members. Yes it's completely free.

You May Also Like