AWS Extends Machine Learning for Easier Deployment & Reduced Cost
SEATTLE -- Today at AWS re:Invent, Amazon Web Services (AWS) announced 13 new machine learning capabilities and services, across all layers in the machine learning stack, to help put machine learning in the hands of even more developers...
Most machine learning models are trained by an algorithm that finds patterns in large amounts of data. The model can then make predictions on new data in a process called "inference." Developers use machine learning frameworks to define these algorithms, train models, and infer predictions. Frameworks (such as TensorFlow, Apache MXNet, and PyTorch) allow developers to design and train sophisticated models, often using multiple GPUs to reduce training times. Most developers use more than one of these frameworks in their day-to-day work. Today, AWS announced significant improvements for developers building with all of these popular frameworks, by improving performance and reducing cost for both training and inference.
- New Amazon Elastic Compute Cloud (EC2) GPU instances (available next week): With eight NVIDIA V100 GPUs, 32GB GPU memory, fast NVMe storage, 96 Intel “Skylake” vCPUs, and 100Gbps networking, the new P3dn.24xl instances are the most powerful machine learning training processors available in the cloud, allowing developers to train models with more data in less time.
- AWS-Optimized TensorFlow framework (generally available today): When training with large amounts of data, developers who choose to use TensorFlow have found that it’s challenging to scale TensorFlow across many GPUs, which often results in low utilization of these GPUs and longer training times for large training jobs. AWS worked on this problem and has innovated on how to make TensorFlow scale across GPUs. By improving the way in which TensorFlow distributes training tasks across those GPUs, the new AWS-Optimized TensorFlow achieves close to linear scalability when training multiple types of neural networks (90 percent efficiency across 256 GPUs, compared to the prior norm of 65 percent). Using the new AWS-Optimized TensorFlow and P3dn instances, developers can now train the popular ResNet-50 model in only 14 minutes, the fastest time recorded, and 50 percent faster than the previous best time. And, these optimizations are generally applicable not just for computer vision models but also for a broader set of deep learning models.
- Amazon Elastic Inference (generally available today): While training rightfully receives a lot of attention, inference actually accounts for the majority of the cost and complexity for running machine learning in production (for every dollar spent on training, nine are spent on inference). Amazon Elastic Inference allows developers to dramatically decrease inference costs with up to 75 percent savings when compared to the cost of using a dedicated GPU instance. Instead of running on a whole Amazon EC2 P2 or P3 instance with relatively low utilization, developers can run on a smaller, general-purpose Amazon EC2 instance and provision just the right amount of GPU performance from Amazon Elastic Inference. Starting at just 1 TFLOP, developers can elastically increase or decrease the amount of inference performance, and only pay for what they use. Elastic Inference supports all popular frameworks, and is integrated with Amazon SageMaker and the Amazon EC2 Deep Learning Amazon Machine Image (AMI). And, developers can start using Amazon Elastic Inference without making any changes to their existing models.
- AWS Inferentia (available in 2019): For larger workloads that consume entire GPUs or require lower latency, AWS announced a high performance machine learning inference chip, custom designed by AWS. AWS Inferentia provides hundreds of teraflops per chip and thousands of teraflops per Amazon EC2 instance for multiple frameworks (including TensorFlow, Apache MXNet, and PyTorch), and multiple data types (including INT-8 and mixed precision FP-16 and bfloat16).
New Amazon SageMaker capabilities make it easier to build, train, and deploy machine learning; developers get hands on with AWS DeepRacer, a 1/18th scale autonomous race car driven by reinforcement learning
Amazon SageMaker is a fully managed service that removes the heavy lifting and guesswork from each step of the machine learning process. Amazon SageMaker makes it easier for developers to build, train, tune, and deploy machine learning models. Today, AWS announced a number of new capabilities for Amazon SageMaker.
- Amazon SageMaker Ground Truth (generally available today): The journey to build machine learning models requires developers to prepare their datasets for training their ML models. Before developers can select their algorithms, build their models, and deploy them to make predictions, human annotators manually review thousands of examples and add the labels required to train machine learning models. This process is time consuming and expensive. Amazon SageMaker Ground Truth makes it much easier for developers to label their data using human annotators through Mechanical Turk, third party vendors, or their own employees. Amazon SageMaker Ground Truth learns from these annotations in real time and can automatically apply labels to much of the remaining dataset, reducing the need for human review. Amazon SageMaker Ground Truth creates highly accurate training data sets, saves time and complexity, and reduces costs by up to up to 70 percent when compared to human annotation.
- AWS Marketplace for Machine Learning (generally available today): Machine learning is moving quickly, with new models and algorithms from academia and industry appearing virtually every week. Amazon SageMaker includes some of the most popular models and algorithms built-in, but to make sure developers continue to have access to the broadest set of capabilities, the new AWS Marketplace for Machine Learning includes over 150 algorithms and models (with more coming every day) that can be deployed directly to Amazon SageMaker. Developers can start using these immediately from SageMaker. Adding a listing to the Marketplace is completely self-service for developers who want to sell through AWS Marketplace.
- Amazon SageMaker RL (generally available today): In machine learning circles, there is a lot of buzz about reinforcement learning because it’s an exciting technology with a ton of potential. Reinforcement learning trains models, without large amounts of training data, and it’s broadly useful when the reward function of a desired outcome is known but the path to achieving it is not and requires a lot of iteration to discover. Healthcare treatments, optimizing manufacturing supply chains, and solving gaming challenges are a few of the areas that reinforcement learning can help address. However, reinforcement learning has a steep learning curve and many moving parts, which effectively puts it out of the reach of all but the most well-funded and technical organizations. Amazon SageMaker RL, the cloud’s first managed reinforcement learning service, allows any developer to build, train, and deploy with reinforcement learning through managed reinforcement learning algorithms, support for multiple frameworks (including Intel Coach and Ray RL), multiple simulation environments (including SimuLink and MatLab), and integration with AWS RoboMaker, AWS’s new robotics service, which provides a simulation platform that integrates well with SageMaker RL.
- AWS DeepRacer (available for pre-order today): In just a few lines of code, developers can start learning about reinforcement learning with AWS DeepRacer, a 1/18th scale fully autonomous race car. The car (with all-wheel drive, monster truck tires, an HD video camera, and on-board compute) is driven using reinforcement learning models trained using Amazon SageMaker. Developers can put their skills to the test and race their cars and models against other developers for prizes and glory in the DeepRacer League, the world’s first global autonomous racing league, open to everyone.
- Amazon SageMaker Neo (generally available today): The new deep learning model compiler lets customers train models once, and run them anywhere with up to 2X improvement in performance. Applications running on connected devices at the edge are particularly sensitive to performance of machine learning models. They require low latency decisions, and are often deployed across a broad number of different hardware platforms. Amazon SageMaker Neo compiles models for specific hardware platforms, optimizing their performance automatically, allowing them to run at up to twice the performance, without any loss in accuracy. As a result, developers no longer need to spend time hand tuning their trained models for each and every hardware platform (saving time and expense). SageMaker Neo supports hardware platforms from NVIDIA, Intel, Xilinx, Cadence, and Arm, and popular frameworks such as TensorFlow, Apache MXNet, and PyTorch. AWS will also make Neo available as an open source project.
Dubbed "America's Un-carrier," T-Mobile is a leading wireless services, products, and service innovation provider. “The AI at T-Mobile team is integrating AI and machine learning into the systems at our customer care centers, enabling our team of experts to serve customers with greater speed and accuracy through Natural Language Understanding models that show them relevant, contextual customer information in real-time," said Matthew Davis, Vice President of IT Development for T-Mobile. "Labeling data has been foundational to creating high performing models, but is also a monotonous task for our data scientists and software engineers. Amazon SageMaker Ground Truth makes the data labeling process easy, efficient, and accessible, freeing up time for them to focus on what they love – building products that deliver the best experiences for our customers and care representatives.”
New AI services bring intelligence to all apps, no machine learning experience required
Many developers want to be able to add intelligent features to their applications without requiring any machine learning experience. Building on existing computer vision, speech, language, and chatbot services, AWS announced a significant expansion of AI services.
- Amazon Textract (available in preview today): Many companies today extract data from documents and forms through manual data entry which is slow and expensive, or using simple optical character recognition (OCR) software, which is often inaccurate and typically produces output that requires extensive post-processing to put the extracted content in a format that is usable by a developer's application. Amazon Textract uses machine learning to instantly read virtually any type of document to accurately extract text and data without the need for any manual review or custom code. Amazon Textract allows developers to quickly automate document workflows, processing millions of document pages in a few hours.
- Amazon Comprehend Medical (generally available today): Building the next generation of medical applications requires being able to understand and analyze the information that is often trapped in free-form, unstructured medical text, such as hospital admission notes or patient medical histories. Comprehend Medical is a highly accurate natural language processing service for medical text, which uses machine learning to extract disease conditions, medications, and treatment outcomes from patient notes, clinical trial reports, and other electronic health records. Comprehend Medical requires no machine learning expertise, no complicated rules to write, no models to train, and it is continuously improving. You pay only for what you use and there are no minimum fees or upfront commitments.
- Amazon Personalize (available in preview today): Based on the same technology that powers Amazon.com, Amazon Personalize is a real-time recommendation and personalization service. Amazon pioneered the use of machine learning for recommendation and personalization over twenty years ago. Experience has shown that there is no master algorithm for personalization. Each use case, from videos, music, products, news articles, has its own specificities, which require a unique mix of data, algorithms, and optimizations. Amazon Personalize provides this experience to customers in a fully managed service, which will build, train, and deploy custom, private personalization and recommendation models for virtually any use case. Amazon Personalize can make recommendations, personalize search results, and segment customers for direct and personalized marketing through email or push notifications.
- Amazon Forecast (available in preview today): Just like personalization, forecasting has traditionally been a bit of a dark art, where customers try to predict future trends in supply chain, inventory levels, and product demand, based on historical data. And just like Amazon Personalize, Amazon Forecast is based on technology that has been developed by Amazon.com and used for a lot of critical forecasting. Forecasting is hard to do well because there are often so many inter-related factors (such as pricing, events, and even the weather). Missing the mark with a forecast can have a significant impact, such as being unable to meet customer demand or significantly over-spending. Amazon Forecast creates accurate time-series forecasts. Using historical data and related causal data, Amazon Forecast will automatically train, tune, and deploy custom, private machine learning forecasting models, so that customers can be more confident that they’ll provide the right customer experience while optimizing their spend.