SDN Technology

Facebook Reinvents Data Center Networking

Not satisfied with just changing how the world socializes, Facebook is setting its sights on remaking data center networking.

During Facebook 's remarkable 11 years from launch to 1.44 billion monthly active users as of the first quarter of this year, the social platform outstripped the capacity of existing networks to carry its traffic. So Facebook had to build four data centers at the heart of a global network, and design its own hardware and software, all to make sure that you don't have to wait a moment to see that photo of your nephew's trombone recital.

Facebook turned to its own designs because the service was moving too fast for networking vendors to keep up.

Rack 'Em Up
Inside the Lulea, Sweden, data center, one of four that Facebook maintains worldwide. [Source: Facebook]
Inside the Lulea, Sweden, data center, one of four that Facebook maintains worldwide.
[Source: Facebook]

"We try to work fast in terms of rolling out new topologies, new speeds, new features and flexibility. We often [find] ourselves waiting and not moving as fast as we want," Omar Baldonado, Facebook manager of the networking team, tells Light Reading.

Facebook doesn't blame vendors. "We totally understand we have special requirements that are different from the rest of their customer base," Baldonado says. "But vendors weren't meeting Facebook needs."

In addition to speed of innovation, Facebook also wants to be able to disaggregate its network -- to mix-and-match vendor-built hardware products with Facebook's own networking software.

Open source hardware design is key to Facebook's networking strategy, Baldonado says. Open source allows Facebook to collaborate with the community of hardware and software engineers outside its own walls, including other large operators such as Rackspace and Microsoft Corp. (Nasdaq: MSFT).

To that end, Facebook co-founded the Open Compute Project (OCP) in 2011, along with Intel Corp. (Nasdaq: INTC); Rackspace ; Goldman Sachs & Co. ; and Andy Bechtolsheim, co-founder of Sun Microsystems Inc. and Arista Networks Inc. and investor in Google (Nasdaq: GOOG) and other companies. The goal: to further open source data center and networking hardware designs.

The Social Network's network
Facebook has four major data centers around the world. The locations are no secret -- they have their own Facebook pages. The first was Prineville, Ore., opened four years ago in April 2011. Others are located in Altoona, Iowa; Forest City, NC; and Luleå, Sweden.

Facebook augments its data centers with a couple of dozen local PoPs to "extend the edge of the network beyond the data centers," Baldonado says. Some 82% of Facebook users are located outside the US, but the majority of data centers are inside the US, Baldonado says.

In designing its own data centers, Facebook realized early on that networking needed to be front and center. The traditional way to design a data center is to lay out racks and cages for the servers first, and then add networking cables in the remaining space, almost as an afterthought, Baldonado says.

Next Page: Weaving a Fabric

1 of 3
Next Page
danielcawrey 6/6/2015 | 1:12:21 PM
Re: Open Compute Project Most people don't realize how network intensive Facebook really is. In order to handle over a billion active users there's a lot of horsepower required.

What I think is most interesting about what Facebook is doing is that it's going to help the next generation of consumer/social/cloud services that arrive. 
Mitch Wagner 6/2/2015 | 9:09:53 PM
Re: Proprietary On-Prem I've avoided Silicon Valley. I take it that's a good show?

I just started watching HOUSE OF CARDS this week. I have a lot of catching up to do. 
Joe Stanganelli 5/31/2015 | 11:09:27 PM
Re: Open Compute Project Of course, Amazon was in a similar situation with its own network and servers -- when Bezos had the bright idea to use those servers to solve the problem of his decreasing margins.  Thus, AWS was born.
Joe Stanganelli 5/31/2015 | 11:07:59 PM
Proprietary On-Prem Facebook's dedication to using (even building) its own hardware and own datacenters has impressed me -- and it was the first thing I thought of after a recent episode of Silicon Valley wherein (SPOILER ALERT) Pied Piper decides to go with on-prem.
Atlantis-dude 5/29/2015 | 9:28:54 AM
Re: Open Compute Project how does their srvr mgmt sw now interface with the netwk? Even if they run their own sw there must be an interface to it. Are they allowing direct access to the bcom interfaces?
bosco_pcs 5/28/2015 | 3:46:34 PM
Re: Open Compute Project A cogent point. White box and open source only enhance interoperability, which is to these folks' benefit. Depending on the weight of their partners, they can also turn their APIs on or off or tweak their algorithms, such as Google's recent announcement of playing favorite to the mobile ready links. So, they are not doing it out of their generosity or sense of fair play - unless it is to their advantage
Mitch Wagner 5/27/2015 | 6:01:35 PM
Re: Open Compute Project The network itself is not the value for Facebook and Google. It's a cost. 
mhhf1ve 5/27/2015 | 2:08:06 PM
Open Compute Project It's really interesting how Google and Facebook have open sourced some of the most vital tools they need to operate, instead of keeping them all proprietary and secret. Sure, Google and Facebook keep their algorithms that actually rake in the revenues secret -- but hardware and software that make their operations more efficient are free for anyone to improve upon. That's a fascinating (and perhaps pivotal) tech culture mindset.
Sign In