Cloud is finally ready, says Spotify VP engineering/infrastructure Nicholas Harteau.

Mitch Wagner, Executive Editor, Light Reading

February 23, 2016

3 Min Read
Spotify Presses Play on Google Cloud

Spotify is moving its massive music streaming infrastructure from on-premises compute to Google Cloud Platform.

Spotify moved from on-premises infrastructure to the cloud because "keeping pace with scaling demands requires ever increasing amounts of focus and effort," writes Nicholas Harteau, Spotify VP engineering and infrastructure, on the company blog Tuesday. "Like good, lazy engineers, we occasionally asked ourselves: Do we really need to do all this stuff?"

"For a long time the answer was 'yes,' " Harteau says. Cloud services were not "at a level of quality, performance and cost that would make cloud a significantly better option for Spotify in the long run. As they say: better the devil you know..."

But cloud has caught up in quality, performance and cost, Harteau says.

Figure 1: Spotify on iPhone Source: Spotify Source: Spotify

Spotify has prior experience with Google's data platform and tools, which help make Spotify's team more effective, Harteau says.

"We have a large and complex backend, so this is a large and complex project that will take us some time to complete," Harteau says.

Spotify runs its products on microservices, several of which are moving from on premises data centers to Google Compute Engine, which uses high IOPS SSD and local SSD storage and autoscaling up and down to meet demand, according to a post on the Google Cloud Platform blog Tuesday morning.

For storage, Spotify is implementing Google Cloud Datastore and Google Cloud Bigtable. Spotify is also deploying Cloud Networking services including Direct Peering, Cloud VPN and Cloud Router to transfer petabytes of data.

Want to know more about the cloud? Visit Light Reading's Cloud Services content channel.

For data, Spotify is changing its technology stack, moving from Hadoop, MapReduce, Hive and "home-grown dashboarding tools" to Google Cloud Pub/Sub, Google Cloud Dataflow, Google BigQuery, Google Cloud Dataproc and other tools, Google says.

Spotify has been a reference customer for Google competitor Amazon Web Services Inc. -- specifically Amazon Simple Storage Service (Amazon S3), using Python-based backend systems to interact with content on S3, and Amazon CloudFront to deliver the application and software updates to users, according to a write-up by AWS. We've got queries in to both Spotify and Amazon on whether that relationship will change.

Another streaming media company, Netflix Inc. (Nasdaq: NFLX), has built its business on the cloud, going all in on AWS and closing the last of its own data centers this month. (See Netflix Cloud Casts Long Shadow Over Cable .)

Related posts:

— Mitch Wagner, Circle me on Google+ Follow me on TwitterVisit my LinkedIn profileFollow me on Facebook, West Coast Bureau Chief, Light Reading. Got a tip about SDN or NFV? Send it to [email protected].

About the Author(s)

Mitch Wagner

Executive Editor, Light Reading

San Diego-based Mitch Wagner is many things. As well as being "our guy" on the West Coast (of the US, not Scotland, or anywhere else with indifferent meteorological conditions), he's a husband (to his wife), dissatisfied Democrat, American (so he could be President some day), nonobservant Jew, and science fiction fan. Not necessarily in that order.

He's also one half of a special duo, along with Minnie, who is the co-habitor of the West Coast Bureau and Light Reading's primary chewer of sticks, though she is not the only one on the team who regularly munches on bark.

Wagner, whose previous positions include Editor-in-Chief at Internet Evolution and Executive Editor at InformationWeek, will be responsible for tracking and reporting on developments in Silicon Valley and other US West Coast hotspots of communications technology innovation.

Beats: Software-defined networking (SDN), network functions virtualization (NFV), IP networking, and colored foods (such as 'green rice').

Subscribe and receive the latest news from the industry.
Join 62,000+ members. Yes it's completely free.

You May Also Like