Spotify Presses Play on Google Cloud
Spotify is moving its massive music streaming infrastructure from on-premises compute to Google Cloud Platform.
Spotify moved from on-premises infrastructure to the cloud because "keeping pace with scaling demands requires ever increasing amounts of focus and effort," writes Nicholas Harteau, Spotify VP engineering and infrastructure, on the company blog Tuesday. "Like good, lazy engineers, we occasionally asked ourselves: Do we really need to do all this stuff?"
"For a long time the answer was 'yes,' " Harteau says. Cloud services were not "at a level of quality, performance and cost that would make cloud a significantly better option for Spotify in the long run. As they say: better the devil you know..."
But cloud has caught up in quality, performance and cost, Harteau says.
Spotify has prior experience with Google's data platform and tools, which help make Spotify's team more effective, Harteau says.
"We have a large and complex backend, so this is a large and complex project that will take us some time to complete," Harteau says.
Spotify runs its products on microservices, several of which are moving from on premises data centers to Google Compute Engine, which uses high IOPS SSD and local SSD storage and autoscaling up and down to meet demand, according to a post on the Google Cloud Platform blog Tuesday morning.
For storage, Spotify is implementing Google Cloud Datastore and Google Cloud Bigtable. Spotify is also deploying Cloud Networking services including Direct Peering, Cloud VPN and Cloud Router to transfer petabytes of data.
For data, Spotify is changing its technology stack, moving from Hadoop, MapReduce, Hive and "home-grown dashboarding tools" to Google Cloud Pub/Sub, Google Cloud Dataflow, Google BigQuery, Google Cloud Dataproc and other tools, Google says.
Spotify has been a reference customer for Google competitor Amazon Web Services Inc. -- specifically Amazon Simple Storage Service (Amazon S3), using Python-based backend systems to interact with content on S3, and Amazon CloudFront to deliver the application and software updates to users, according to a write-up by AWS. We've got queries in to both Spotify and Amazon on whether that relationship will change.
Another streaming media company, Netflix Inc. (Nasdaq: NFLX), has built its business on the cloud, going all in on AWS and closing the last of its own data centers this month. (See Netflix Cloud Casts Long Shadow Over Cable .)
- Google to Weave Fiber to Huntsville, Ala.
- Ericsson CEO on Amazon, Google Partnerships
- Google Opening $600M Tennessee Data Center
- Google Makes VMware Co-Founder Cloud Boss
- Google's 5G Radio Ambitions Are Expanding
- Google Will Accelerate Fiber, Cloud in 2016
- Alphabet Is Serious About Google Fiber