4 Fantastic OpenStack Lessons From Yahoo

Mitch Wagner
4/25/2017
50%
50%

OpenStack's and Yahoo's reputations are both taking a beating lately. Critics say the same thing about both: that they've failed to live up to their promise.

And yet, while Yahoo disappoints Wall Street, and has suffered security problems, a billion users -- literally a billion -- rely on the company's Mail, Finance, Sports and other services. (See US Indictment Says Russian Spies Were Behind Yahoo Hack, Yahoo's Marissa Mayer Gets $23M Kiss-Off, Another Hack Announced by Yahoo and Verizon Knocks $350M off the Price of Yahoo.)

And Yahoo relies on OpenStack to deliver those services. Which should come as no surprise -- even OpenStack's staunchest critics agree it runs well at hyperscale; where OpenStack has failed, they say, is in its initial vision of providing a platform that can replace Amazon Web Services Inc. (AWS) for most enterprises, who lack the resources of hyperscale providers, like Yahoo. (Oddly, OpenStack defenders say the same thing, choosing to emphasize the amount of liquid in the glass rather than its half-emptiness.) (See VMware Damns OpenStack With Faint Praise, Rackspace: OpenStack's Death Is #FakeNews, Mirantis Pivots as OpenStack Loses 'Wow Factor' and OpenStack: Small Pond, but the Big Fish Love It.)

Yahoo shared its OpenStack lessons in a recent blog post -- on Tumblr, naturally, which Yahoo owns and operates: Operating OpenStack at Scale.

Among the lessons Yahoo says it learned:

A successful private cloud needs to hide complexity
Users and developers shouldn't have to think about what's going on with the infrastructure, according to the blog post signed by James Penick, cloud architect and Gurpreet Kaur, product manager, for Yahoo.

"It must simultaneously handle constant organic traffic growth, unanticipated spikes, a multitude of hardware vendors, and discordant customer demands," Yahoo says. "The depth of this complexity only increases with the age of the business, leaving a private cloud operator saddled with legacy hardware, old network infrastructure, customers dependent on legacy operating systems, and the list goes on. These are the foundations of the horror stories told by grizzled operators around the campfire."

Provide infrastructure-as-a-service from a centralized team
"An a-la-carte-IaaS, where each user is expected to manage their own control plane and inventory, just isn't sustainable at scale," Yahoo says. Multiple teams duplicate effort and "removes the opportunity for improved synergy with all levels of the business." A single, centralized team collaborates with the supply chain, data center site operations, finance, and the engineering teams who are the IaaS customers.


M&A activity is turning the cloud upside down. Find out what you need to know in our special report: Mergers, Acquisitions & IPOs Are Rocking the Cloud.


Developers love instant access to compute resources
"It empowers our engineers to innovate, prototype, develop, and quickly iterate on ideas," Yahoo says. "No longer is a developer reliant on a static and costly development machine under their desk."

Start at the bottom and think about the underlying hardware
Think about the hardware needs for the services you're running. For example, databases have intense I/O needs. Sounds obvious, right? "Yet many deployers skimp on the hardware," Yahoo says. "The performance of the whole cluster is bottlenecked by the DB I/O. By thinking ahead you can save yourself a lot of heartburn later on."

Yahoo shares plenty more insights and lessons in its post, which you can read here: Operating OpenStack at Scale.

— Mitch Wagner Follow me on Twitter Visit my LinkedIn profile Visit my blog Friend me on Facebook Editor, Enterprise Cloud News

(31)  | 
Comment  | 
Print  | 
Oldest First  |  Newest First  |  Threaded View        ADD A COMMENT
Page 1 / 4   >   >>
mhhfive
50%
50%
mhhfive,
User Rank: Light Sabre
4/25/2017 | 11:16:15 AM
Nice to see Yahoo is so open about its experience...
It sounds like a lot of these lessons are the same as Google's behind the scenes tech, but less of it is home-grown, as Google's datacenters are. It will certainly be interesting to hear how Verizon integrates Yahoo and AOL's operations going forward, and I hope they keep up this kind of transparency from their engineering teams.
maryam@impact
50%
50%
[email protected],
User Rank: Light Sabre
4/25/2017 | 11:51:01 AM
Re: Nice to see Yahoo is so open about its experience...
I agree, the open perspective is refreshing because it will hopefully allow other organizations to avoid their issues. Yahoo did miss the opportunity to capitalize on their base they started out as an innovator and quickly aged they remind me of Palm and Aol that had an incredible market advantage and could not innovate with the market. It is a lesson learned for the startups of today innovation is constant and resting on your laurels for even a moment is deadly.

 
JohnMason
50%
50%
JohnMason,
User Rank: Light Sabre
4/25/2017 | 12:13:02 PM
Verizon effect
I understand that the Verizon acquisition of Yahoo, and Yahoo's transformation into the new company "Oath," may affect OpenStack down the road. Today I discovered from a colleague that his Verizon email account was being closed because, according to him, "Verizon is getting out of the email business." What would that mean for Yahoo-Oath, and OpenStack, if anything?
Scott_Ferguson
50%
50%
Scott_Ferguson,
User Rank: Light Beer
4/25/2017 | 12:20:23 PM
Re: Verizon effect
@John: That's a great point about the Verizon acquisition and if/how they use OpenStack. Verizon still runs its own cloud, correct? I wonder if they use OpenStack in anyway? I also know that the Verizon cloud is alledgedly up for sale, so does Yahoo go with that and then does the OpenStack stuff goes as well?
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Light Sabre
4/25/2017 | 1:45:31 PM
Hiding & a-la-carte
"A successful private cloud needs to hide complexity"

 

"Hide" probably isn't the best word here (although some may indicate actually advocate that very meaning).  "Set aside," to my mind, is better.  You want that added functionality readily accessible as "advanced options" or whatnot.  The problem with a lot of GUI-based solutions, alas, is that they tend to be inflexibly one-size-fits-all -- leading to back-end problems and coders getting around those problems with undocumented backdoors.  The result: A messy, fundamentally broken infrastructure.

As for a-la-carte IaaS, I agree.  This sort of thing -- whether applied to IaaS or something else -- tends to lead to "Shadow IT" solutions and the like (again, leading to a messy, fundamentally broken infrastructure).
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Light Sabre
4/25/2017 | 1:46:49 PM
Re: Verizon effect
Am I the only one seeing any significance in the notion that, when you abbreviate "OpenStack" to "OS," the pronounciation of that with a lisp becomes "Oath"?  ;)
mhhfive
50%
50%
mhhfive,
User Rank: Light Sabre
4/25/2017 | 2:52:04 PM
Re: Nice to see Yahoo is so open about its experience...
AOL admittedly had a built-in expiration date because it relied on dialup technology and there was no practical path to offering DSL even at its peak. The investment necessary for AOL to become an ISP that could handle broadband would have been prohibitively expensive. AOL did what it had to do at the time (with Time Warner). I'm not sure how AOL could have done better? It still exists! Yahoo, on the other hand, had so many chances to revive itself... and it just couldn't overcome its own inertia.
mhhfive
50%
50%
mhhfive,
User Rank: Light Sabre
4/25/2017 | 3:00:41 PM
Re: Verizon effect
I'm not sure Oath is such a great name but if you like it for whatever reasons then I guess it's better than sticking with AOL. I think at this point AOL doesn't have much positive connotation. It's basically email for old people now. And Yahoo is headed down the same path.
danielcawrey
50%
50%
danielcawrey,
User Rank: Light Sabre
4/25/2017 | 3:52:38 PM
Re: Verizon effect
It's really sad to see what's happening with Yahoo and OpenStack. 

I actually had no idea Yahoo's services used the platform. If the company could have leveraged OpenStack into a power player for open systems, that would have been a great advantage. Sadly, this is not what has happened. 
mhhfive
50%
50%
mhhfive,
User Rank: Light Sabre
4/25/2017 | 6:12:00 PM
Re: Verizon effect
> "... the Verizon cloud is alledgedly up for sale, so does Yahoo go with that and then does the OpenStack stuff goes as well?"

It will be interesting to see what happens to all the Yahoo properties that don't fit nicely under Verizon's umbrella. Will Tumblr shut down? Where does Flickr go? Yahoo acquired a LOT of startups over its lifetime, and it's not clear how any of them really fit with the company now or with its future parent company. 

Yahoo has a lot of datacenter capacity that absorbed a lot of these web services, but if Verizon doesn't have the same kind of appetite for trying to scale up web startups, what's going to happen to Yahoo's datacenters? Will Verizon try to spin off Yahoo's datacenter operations like an AWS wannabe? Or is the cloud industry already too cluttered for yet another cloud service the size of Yahoo? 
Page 1 / 4   >   >>
Featured Video
Flash Poll
Upcoming Live Events
September 17-19, 2019, Dallas, Texas
October 1-2, 2019, New Orleans, Louisiana
October 10, 2019, New York, New York
October 22, 2019, Los Angeles, CA
November 5, 2019, London, England
November 7, 2019, London, UK
December 3, 2019, New York, New York
December 3-5, 2019, Vienna, Austria
March 16-18, 2020, Embassy Suites, Denver, Colorado
May 18-20, 2020, Irving Convention Center, Dallas, TX
All Upcoming Live Events