4 Fantastic OpenStack Lessons From Yahoo

Mitch Wagner
4/25/2017
50%
50%

OpenStack's and Yahoo's reputations are both taking a beating lately. Critics say the same thing about both: that they've failed to live up to their promise.

And yet, while Yahoo disappoints Wall Street, and has suffered security problems, a billion users -- literally a billion -- rely on the company's Mail, Finance, Sports and other services. (See US Indictment Says Russian Spies Were Behind Yahoo Hack, Yahoo's Marissa Mayer Gets $23M Kiss-Off, Another Hack Announced by Yahoo and Verizon Knocks $350M off the Price of Yahoo.)

And Yahoo relies on OpenStack to deliver those services. Which should come as no surprise -- even OpenStack's staunchest critics agree it runs well at hyperscale; where OpenStack has failed, they say, is in its initial vision of providing a platform that can replace Amazon Web Services Inc. (AWS) for most enterprises, who lack the resources of hyperscale providers, like Yahoo. (Oddly, OpenStack defenders say the same thing, choosing to emphasize the amount of liquid in the glass rather than its half-emptiness.) (See VMware Damns OpenStack With Faint Praise, Rackspace: OpenStack's Death Is #FakeNews, Mirantis Pivots as OpenStack Loses 'Wow Factor' and OpenStack: Small Pond, but the Big Fish Love It.)

Yahoo shared its OpenStack lessons in a recent blog post -- on Tumblr, naturally, which Yahoo owns and operates: Operating OpenStack at Scale.

Among the lessons Yahoo says it learned:

A successful private cloud needs to hide complexity
Users and developers shouldn't have to think about what's going on with the infrastructure, according to the blog post signed by James Penick, cloud architect and Gurpreet Kaur, product manager, for Yahoo.

"It must simultaneously handle constant organic traffic growth, unanticipated spikes, a multitude of hardware vendors, and discordant customer demands," Yahoo says. "The depth of this complexity only increases with the age of the business, leaving a private cloud operator saddled with legacy hardware, old network infrastructure, customers dependent on legacy operating systems, and the list goes on. These are the foundations of the horror stories told by grizzled operators around the campfire."

Provide infrastructure-as-a-service from a centralized team
"An a-la-carte-IaaS, where each user is expected to manage their own control plane and inventory, just isn't sustainable at scale," Yahoo says. Multiple teams duplicate effort and "removes the opportunity for improved synergy with all levels of the business." A single, centralized team collaborates with the supply chain, data center site operations, finance, and the engineering teams who are the IaaS customers.


M&A activity is turning the cloud upside down. Find out what you need to know in our special report: Mergers, Acquisitions & IPOs Are Rocking the Cloud.


Developers love instant access to compute resources
"It empowers our engineers to innovate, prototype, develop, and quickly iterate on ideas," Yahoo says. "No longer is a developer reliant on a static and costly development machine under their desk."

Start at the bottom and think about the underlying hardware
Think about the hardware needs for the services you're running. For example, databases have intense I/O needs. Sounds obvious, right? "Yet many deployers skimp on the hardware," Yahoo says. "The performance of the whole cluster is bottlenecked by the DB I/O. By thinking ahead you can save yourself a lot of heartburn later on."

Yahoo shares plenty more insights and lessons in its post, which you can read here: Operating OpenStack at Scale.

— Mitch Wagner Follow me on Twitter Visit my LinkedIn profile Visit my blog Friend me on Facebook Editor, Enterprise Cloud News

(31)  | 
Comment  | 
Print  | 
Threaded  |  Newest First  |  Oldest First        ADD A COMMENT
mhhfive
50%
50%
mhhfive,
User Rank: Light Sabre
4/25/2017 | 11:16:15 AM
Nice to see Yahoo is so open about its experience...
It sounds like a lot of these lessons are the same as Google's behind the scenes tech, but less of it is home-grown, as Google's datacenters are. It will certainly be interesting to hear how Verizon integrates Yahoo and AOL's operations going forward, and I hope they keep up this kind of transparency from their engineering teams.
maryam@impact
50%
50%
[email protected],
User Rank: Light Sabre
4/25/2017 | 11:51:01 AM
Re: Nice to see Yahoo is so open about its experience...
I agree, the open perspective is refreshing because it will hopefully allow other organizations to avoid their issues. Yahoo did miss the opportunity to capitalize on their base they started out as an innovator and quickly aged they remind me of Palm and Aol that had an incredible market advantage and could not innovate with the market. It is a lesson learned for the startups of today innovation is constant and resting on your laurels for even a moment is deadly.

 
mhhfive
50%
50%
mhhfive,
User Rank: Light Sabre
4/25/2017 | 2:52:04 PM
Re: Nice to see Yahoo is so open about its experience...
AOL admittedly had a built-in expiration date because it relied on dialup technology and there was no practical path to offering DSL even at its peak. The investment necessary for AOL to become an ISP that could handle broadband would have been prohibitively expensive. AOL did what it had to do at the time (with Time Warner). I'm not sure how AOL could have done better? It still exists! Yahoo, on the other hand, had so many chances to revive itself... and it just couldn't overcome its own inertia.
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Light Sabre
4/29/2017 | 7:00:56 PM
Re: Nice to see Yahoo is so open about its experience...
> AOL admittedly had a built-in expiration date because it relied on dialup technology

This seems like a dubious proposition to me, considering EarthLink (now Windstream) -- among many other providers -- was based on dialup, and has managed to pivot quite well into cutting-edge techs like SDWAN.
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Light Sabre
4/29/2017 | 7:05:28 PM
Re: Nice to see Yahoo is so open about its experience...
@maryam: They didn't so much "age" as "got old."

But now I'm speaking (er, typing) in riddles.

They went from startup to big company -- and started acting like a big company.  Slow.  Lumbering.  Bureaucratic.

Google/Alphabet, meanwhile, kicked their butts because, despite having gone from startup to MONOLITH, they still act like a startup -- constantly innovating.
maryam@impact
50%
50%
[email protected],
User Rank: Light Sabre
4/29/2017 | 7:07:15 PM
Re: Nice to see Yahoo is so open about its experience...
LOL very true the lack of innovation it what ages people and companies alike!
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Light Sabre
4/29/2017 | 7:10:53 PM
Re: Nice to see Yahoo is so open about its experience...
@maryam: Indeed.  The "got old" language/analogy was in my mind because I recently rewatched part of one of my favorite films: Heist (the one with Gene Hackman, written and directed by David Mamet).   There's a great line Delroy Lindo's character has at the end of an argument about the other character making a giant blunder where he says, "Baby, you got old."
maryam@impact
50%
50%
[email protected],
User Rank: Light Sabre
4/29/2017 | 11:56:31 PM
Re: Nice to see Yahoo is so open about its experience...
Joe very appropriate for the tech generation we live in because everything gets old in about 10 minutes!
JohnMason
50%
50%
JohnMason,
User Rank: Light Sabre
4/25/2017 | 12:13:02 PM
Verizon effect
I understand that the Verizon acquisition of Yahoo, and Yahoo's transformation into the new company "Oath," may affect OpenStack down the road. Today I discovered from a colleague that his Verizon email account was being closed because, according to him, "Verizon is getting out of the email business." What would that mean for Yahoo-Oath, and OpenStack, if anything?
Scott_Ferguson
50%
50%
Scott_Ferguson,
User Rank: Light Beer
4/25/2017 | 12:20:23 PM
Re: Verizon effect
@John: That's a great point about the Verizon acquisition and if/how they use OpenStack. Verizon still runs its own cloud, correct? I wonder if they use OpenStack in anyway? I also know that the Verizon cloud is alledgedly up for sale, so does Yahoo go with that and then does the OpenStack stuff goes as well?
mhhfive
50%
50%
mhhfive,
User Rank: Light Sabre
4/25/2017 | 6:12:00 PM
Re: Verizon effect
> "... the Verizon cloud is alledgedly up for sale, so does Yahoo go with that and then does the OpenStack stuff goes as well?"

It will be interesting to see what happens to all the Yahoo properties that don't fit nicely under Verizon's umbrella. Will Tumblr shut down? Where does Flickr go? Yahoo acquired a LOT of startups over its lifetime, and it's not clear how any of them really fit with the company now or with its future parent company. 

Yahoo has a lot of datacenter capacity that absorbed a lot of these web services, but if Verizon doesn't have the same kind of appetite for trying to scale up web startups, what's going to happen to Yahoo's datacenters? Will Verizon try to spin off Yahoo's datacenter operations like an AWS wannabe? Or is the cloud industry already too cluttered for yet another cloud service the size of Yahoo? 
JohnMason
50%
50%
JohnMason,
User Rank: Light Sabre
4/28/2017 | 11:59:03 AM
Re: Verizon effect
"is the cloud industry already too cluttered for yet another cloud service the size of Yahoo"

It may be cluttered but I don't know if it is TOO cluttered for yet another cloud service. Aren't these the salad days for the industry? That is, isn't the market still unsaturated, and still (fairly) new?
kq4ym
50%
50%
kq4ym,
User Rank: Light Sabre
5/5/2017 | 12:25:57 PM
Re: Verizon effect
Yes, interesting to watch just what Verizon may do with all the parts and pieces of Yahoo. It would not be surprising to see some offloading of assets not in Verizon's area of expertise yet maybe those can just be run on "auto pilot" and just see how the profits proceed or not in coming years?
JohnMason
50%
50%
JohnMason,
User Rank: Light Sabre
4/28/2017 | 12:03:10 PM
Re: Verizon effect
@Scott: About Verizon using OpenStack, this is from their website:

http://www.verizon.com/about/news/verizon-launches-industry-leading-large-openstack-nfv-deployment

"Verizon has completed the industry's largest known Network Function Virtualization OpenStack cloud deployment across five of its U.S. data centers."

 

 
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Light Sabre
4/29/2017 | 7:09:03 PM
Re: Verizon effect
@John: Yes, VZ collaborated with Red Hat on that over the past year.  The attraction is as an extension of their corporate network (particuarly where mobile VPN is concerned).  Plus, almost all the big companies are getting involved in the open-source community these days because they realize that it's simply good for business to be in a room with a bunch of other smart people.
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Light Sabre
4/25/2017 | 1:46:49 PM
Re: Verizon effect
Am I the only one seeing any significance in the notion that, when you abbreviate "OpenStack" to "OS," the pronounciation of that with a lisp becomes "Oath"?  ;)
mhhfive
50%
50%
mhhfive,
User Rank: Light Sabre
4/25/2017 | 3:00:41 PM
Re: Verizon effect
I'm not sure Oath is such a great name but if you like it for whatever reasons then I guess it's better than sticking with AOL. I think at this point AOL doesn't have much positive connotation. It's basically email for old people now. And Yahoo is headed down the same path.
danielcawrey
50%
50%
danielcawrey,
User Rank: Light Sabre
4/25/2017 | 3:52:38 PM
Re: Verizon effect
It's really sad to see what's happening with Yahoo and OpenStack. 

I actually had no idea Yahoo's services used the platform. If the company could have leveraged OpenStack into a power player for open systems, that would have been a great advantage. Sadly, this is not what has happened. 
maryam@impact
50%
50%
[email protected],
User Rank: Light Sabre
4/26/2017 | 12:28:55 PM
Re: Verizon effect
mhhfive very funny but there are many people who were early internet adopters who had AOL and Yahoo accounts because they were the only providers at the time. I still have an AOL account because it's so old that unhooking it from everything and everyone would be time consumptive but I also have a google account because it has many more advantages.

I don't think Verizon will shut down properties it will just not be as smooth of an integration, like so many acquisitions.
mhhfive
50%
50%
mhhfive,
User Rank: Light Sabre
4/26/2017 | 12:55:51 PM
Re: Verizon effect
I think Verizon will try to sell off Yahoo!'s properties like Flickr that don't quite fit with Verizon's other brands and services. Maybe Tumnlr will survive alongside AOL's media blogs in some way to prove Oath's ad network for attracting eyeballs and getting engagement.
maryam@impact
50%
50%
[email protected],
User Rank: Light Sabre
4/28/2017 | 11:07:50 AM
Re: Verizon effect
They might if there is a buyer but some properties may just die a natural technology death because the market competitors are stronger and better positioned time will tell.
mhhfive
50%
50%
mhhfive,
User Rank: Light Sabre
4/28/2017 | 11:40:41 AM
Re: Verizon effect
I'm sure some Yahoo properties will die. Yahoo!'s streaming media efforts are already dying or dead. I think Katie Couric has moved on. Community had a rabid fan base that just didn't follow the show to Yahoo!'s streaming video site which was slow and laggy and required an annoying app download.
maryam@impact
50%
50%
[email protected],
User Rank: Light Sabre
4/28/2017 | 12:12:37 PM
Re: Verizon effect
 

I am sure they anticipated the dud technology when they priced the acquisition some of it was tech based some of it was just for the access to base they had acquired. Time will be the test of how it plays out in the marketplace and if they synergy is achieved.
Michelle
50%
50%
Michelle,
User Rank: Light Sabre
4/28/2017 | 2:51:14 PM
Re: Verizon effect
I don't see a new life for Yahoo. So much has gone wrong for them over the years, I don't see how they could recover. The brand has suffered much.
mhhfive
50%
50%
mhhfive,
User Rank: Light Sabre
4/28/2017 | 4:04:37 PM
Re: Verizon effect
The same could have been said for AOL and it's still sorta alive and kicking. Yahoo isn't dead yet. But it's close, it's like Apple in 1997.
maryam@impact
50%
50%
[email protected],
User Rank: Light Sabre
4/28/2017 | 10:05:43 PM
Re: Verizon effect
It will probably not see the resurrection that Apple saw but some of its tech may live on.
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Light Sabre
4/29/2017 | 7:12:27 PM
Re: Verizon effect
> It will probably not see the resurrection that Apple saw

Yeah, that's kind of a one-in-a-million scenario...  There's only one Steve Jobs.

Although Jeff Bezos (or, arguably, Travis Kalanick) could be the next one, so to speak.
maryam@impact
50%
50%
[email protected],
User Rank: Light Sabre
4/29/2017 | 11:54:22 PM
Re: Verizon effect
True Jobs was an innovator but even he had dry patches before he resurrected Apple again, there may be another Bezos or Jobs out there right now or in the future so who knows what the next wave of tech innovation will bring.
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Light Sabre
4/29/2017 | 7:13:26 PM
Re: Verizon effect
@Michelle: I don't think Verizon bought Yahoo so much for the brand as for a bunch of other stuff.

Yahoo lived by the acqui-hire -- and they died by the acqui-hire.  ;)
Michelle
50%
50%
Michelle,
User Rank: Light Sabre
4/29/2017 | 8:35:35 PM
Re: Verizon effect
@Joe Interesting. There's been so much coverage of security breaches and aquisition pricing that I've missed the real reasons Verizon might want Yahoo. I was very surprised they went ahead with the deal after the security breaches.
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Light Sabre
4/25/2017 | 1:45:31 PM
Hiding & a-la-carte
"A successful private cloud needs to hide complexity"

 

"Hide" probably isn't the best word here (although some may indicate actually advocate that very meaning).  "Set aside," to my mind, is better.  You want that added functionality readily accessible as "advanced options" or whatnot.  The problem with a lot of GUI-based solutions, alas, is that they tend to be inflexibly one-size-fits-all -- leading to back-end problems and coders getting around those problems with undocumented backdoors.  The result: A messy, fundamentally broken infrastructure.

As for a-la-carte IaaS, I agree.  This sort of thing -- whether applied to IaaS or something else -- tends to lead to "Shadow IT" solutions and the like (again, leading to a messy, fundamentally broken infrastructure).
Featured Video
Flash Poll
Upcoming Live Events
September 17-19, 2019, Dallas, Texas
October 1-2, 2019, New Orleans, Louisiana
October 10, 2019, New York, New York
October 22, 2019, Los Angeles, CA
November 5, 2019, London, England
November 7, 2019, London, UK
December 3, 2019, New York, New York
December 3-5, 2019, Vienna, Austria
March 16-18, 2020, Embassy Suites, Denver, Colorado
May 18-20, 2020, Irving Convention Center, Dallas, TX
All Upcoming Live Events