AWS Announces Five New Amazon SageMaker Capabilities for Scaling with Models
At AWS re:Invent, Amazon Web Services, Inc. (AWS), an Amazon.com, Inc. company (NASDAQ: AMZN), today announced five new capabilities within Amazon SageMaker to help accelerate building, training, and deployment of large language models and other foundation models. As models continue to transform customer experiences across industries, SageMaker is making it easier and faster for organizations to build, train, and deploy machine learning (ML) models that power a variety of generative AI uses cases. However, to use models successfully, customers need advanced capabilities that efficiently manage model development, usage, and performance. That's why most industry leading models such as Falcon 40B and 180B, IDEFICS, Jurassic-2, Stable Diffusion, and StarCoder are all trained on SageMaker. Today's announcements include a new capability that further enhances SageMaker for scaling with models by accelerating model training time. Another new SageMaker capability optimizes managed ML infrastructure operations by reducing deployment costs and latency of models. AWS is also introducing a new SageMaker Clarify capability that makes it easier to select the right model based on quality parameters that support responsible use of AI. To help customers apply these models across organizations, AWS is also introducing a new no-code capability in SageMaker Canvas that makes it faster and easier for customers to prepare data using natural-language instructions. Additionally, SageMaker Canvas continues to democratize model building and customization by making it easier for customers to use models to extract insights, make predictions, and generate content using an organization's proprietary data. These advancements build on SageMaker's extensive capabilities to help customers innovate with ML at scale. To get started with Amazon SageMaker, visit aws.amazon.com/sagemaker.
Recent advancements in ML, along with ready availability of scalable compute capacity and the massive proliferation of data, have led to the rise of models that contain billions of parameters, making them capable of performing a wide range of tasks like writing blog posts, generating images, solving math problems, engaging in dialog, and answering questions based on a document. Today, tens of thousands of customers like 3M, AstraZeneca, Ferrari, LG AI Research, RyanAir, Thomson Reuters, and Vanguard are using SageMaker to make more than 1.5 trillion inference requests every month. In addition, customers like AI21 Labs, Stability AI, and Technology Innovation Institute are using SageMaker to train models with up to billions of parameters. As customers move from building mostly task-specific models to the large, general-purpose models that power generative AI, they work with massive datasets and more complex infrastructure setups-all while optimizing for cost and performance. Customers also want to build and customize their own models to create unique customer experiences, embodying the company's voice, style, and services. With more than 380 capabilities and features added since the service was launched in 2017, SageMaker offers customers everything they need to build, train, and deploy production-ready models at scale.
"Machine learning is one of the most profound technological developments in recent history, and interest in models has spread to every organization," said Bratin Saha, vice president of Artificial Intelligence and Machine Learning at AWS. "This growth in interest is presenting new scaling challenges for customers who want to build, train, and deploy models faster. From accelerating training, optimizing hosting costs, reducing latency, and simplifying the evaluation of foundation models, to expanding our no-code model-building capabilities, we are on a mission to democratize access to high-quality, cost-efficient machine learning models for organizations of all sizes. With today's announcements, we are enhancing Amazon SageMaker with fully managed, purpose-built capabilities that help customers make the most of their machine learning investments."
New capabilities make it easier and faster for customers to train and operate models to power their generative AI applications
As generative AI continues to gain momentum, many emerging applications will rely on models. But most organizations struggle to adapt their infrastructure to meet the demands of these new models, which can be difficult to train and operate efficiently at scale. Today, SageMaker is adding two new capabilities that help ease the burdens of training and deploying models at scale.
Today, customers have a wide range of options when choosing a model to power their generative AI applications, and they want to compare these models quickly to find the best option based on relevant quality and responsible AI parameters (e.g., accuracy, fairness, and robustness). However, when comparing models that perform the same function (e.g., text generation or summarization) or that are within the same family (e.g., Falcon 40B versus Falcon 180B), each model will perform differently across various responsible AI parameters. Even the same model fine-tuned on two different datasets could perform differently, making it challenging to know which version works best. To start comparing models, organizations must first spend days identifying relevant benchmarks, setting up evaluation tools, and running assessments on each model. While customers have access to publicly available model benchmarks, they are often unable to evaluate the performance of models on prompts that are representative of their specific use cases. In addition, these benchmarks are often hard to decipher and are not useful for evaluating criteria like brand voice, relevance, and style. Then an organization has to go through the time-consuming process of manually analyzing results, and repeating this process for every new use case or fine-tuned model.
SageMaker Clarify now helps customers evaluate, compare, and select the best models for their specific use case based on their chosen parameters to support an organization's responsible use of AI. With the new capability in SageMaker Clarify, customers can easily submit their own model for evaluation or select a model via SageMaker JumpStart. In SageMaker Studio, customers choose the models that they want to compare for a given task, such as question answering or content summarization. Customers then select evaluation parameters and upload their own prompt dataset or select from built-in, publicly available datasets. For sensitive criteria or nuanced content that requires sophisticated human judgement, customers can choose to use their own workforce, or a managed workforce provided by SageMaker Ground Truth, to review the responses within minutes using feedback mechanisms. Once customers finish the setup process, SageMaker Clarify runs its evaluations and generates a report, so customers can quickly evaluate, compare, and select the best model based on performance criteria.
New Amazon SageMaker Canvas enhancements make it easier and faster for customers to integrate generative AI into their workflows
Amazon SageMaker Canvas helps customers build ML models and generate predictions without writing a single line of code. Today's announcement expands on SageMaker Canvas' existing, ready-to-use capabilities that help customers use models to power a range of use cases, in a no-code environment.
Hugging Face is a leading machine learning company and open platform for AI builders, offering open foundation models, and the tools to create them. "Hugging Face has been using SageMaker HyperPod to create important new open foundation models like StarCoder, IDEFICS, and Zephyr, which have been downloaded millions of times," said Jeff Boudier, head of Product at Hugging Face. "SageMaker HyperPod's purpose-built resiliency and performance capabilities have enabled our open science team to focus on innovating and publishing important improvements to the ways foundation models are built, rather than managing infrastructure. We especially liked how SageMaker HyperPod is able to detect ML hardware failure and quickly replace the faulty hardware without disrupting ongoing model training. Because our teams need to innovate quickly, this automated job recovery feature helped us minimize disruption during the foundation model training process, helping us save hundreds of hours of training time in just a year."
Salesforce is a leading AI customer relationship management (CRM) platform, driving productivity and trusted customer experiences powered by data, AI, and CRM. "At Salesforce, we have an open ecosystem approach to foundation models, and Amazon SageMaker is a vital component, helping us scale our architecture and accelerate our go-to-market," said Bhavesh Doshi, vice president of Engineering at Salesforce. "Using the new SageMaker Inference capability, we were able to put all our models onto a single SageMaker endpoint that automatically handled all the resource allocation and sharing of the compute resources, accelerating performance and reducing deployment cost of foundation models."
Thomson Reuters is a leading source of information, including one of the world's most trusted news organizations. "One of the challenges that our engineers face is managing customer call resources during peak seasons to ensure the optimal number of customer service personnel are hired to handle the influx of inquiries," said Maria Apazoglou, vice president of Artificial Intelligence, Business Intelligence and Data Platforms at Thomson Reuters. "Historical analysis of call center data containing call volume, wait time, date, and other relevant metrics is time consuming. Our teams are leveraging the new data preparation and customization capabilities in SageMaker Canvas to train models on company data to identify patterns and trends that impact call volume during peak hours. It was extremely easy for us to build ML models using our own data, and we look forward to increasing the use of foundational models-without writing any code-through Canvas."
Workday, Inc. is a cloud-based software vendor specializing in human capital management (HCM) and financial management applications. "More than 10,000 organizations around the world rely on Workday to manage their most valuable assets-their people and their money," said Shane Luke, vice president of AI and Machine Learning at Workday. "We provide responsible and transparent solutions to customers by selecting the best foundation model that reflects our company's policies around the responsible use of AI. For tasks such as creating job descriptions, which must be high quality and promote equal opportunity, we tested the new model evaluation capability in Amazon SageMaker and are excited about the ability to measure foundation models across metrics such as bias, quality, and performance. We look forward to using this service in the future to compare and select models that align with our stringent responsible AI criteria."
About Amazon Web Services
Since 2006, Amazon Web Services has been the world's most comprehensive and broadly adopted cloud. AWS has been continually expanding its services to support virtually any workload, and it now has more than 240 fully featured services for compute, storage, databases, networking, analytics, machine learning and artificial intelligence (AI), Internet of Things (IoT), mobile, security, hybrid, virtual and augmented reality (VR and AR), media, and application development, deployment, and management from 102 Availability Zones within 32 geographic regions, with announced plans for 15 more Availability Zones and five more AWS Regions in Canada, Germany, Malaysia, New Zealand, and Thailand. Millions of customers-including the fastest-growing startups, largest enterprises, and leading government agencies-trust AWS to power their infrastructure, become more agile, and lower costs. To learn more about AWS, visit aws.amazon.com.
Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. Amazon strives to be Earth's Most Customer-Centric Company, Earth's Best Employer, and Earth's Safest Place to Work. Customer reviews, 1-Click shopping, personalized recommendations, Prime, Fulfillment by Amazon, AWS, Kindle Direct Publishing, Kindle, Career Choice, Fire tablets, Fire TV, Amazon Echo, Alexa, Just Walk Out technology, Amazon Studios, and The Climate Pledge are some of the things pioneered by Amazon. For more information, visit amazon.com/about and follow @AmazonNews.