TMCnet News
Anyscale Launches Aviary: Open Source Infrastructure to Simplify Large Language Model DeploymentSAN FRANCISCO, May 31, 2023 (GLOBE NEWSWIRE) -- Anyscale, the AI infrastructure company built by the creators of Ray, the world’s fastest growing open source unified framework for scalable computing, today launched Aviary, a new open source project designed to help developers simplify the painstaking process of choosing and integrating the best open source large language models (LLMs) into their applications. Once the best model is selected for the application, Aviary makes the transition to production effortless. “Our goal is to ensure that any developer can integrate AI into their products and to make it easy to develop, scale, and productionize AI applications without building and managing infrastructure. With the Aviary project, we are giving developers the tools to leverage LLMs in their applications,” said Robert Nishihara, Co-founder and CEO of Anyscale. “AI is moving so rapidly that many companies are finding that their infrastructure choices prevent them from taking advantage of the latest LLM capabilities. They need access to a platform that lets them leverage the entire open source LLM ecosystem in a future-proof, performant and cost-effective manner.” The AI Imperative Gartner® Research writes, “By 2026, 75% of newly developed enterprise applications will incorporate AI- or ML-based models, up from less than 5% in 2023.”1 Selecting the right open source LLM for an application is a complex process and requires significant expertise in machine learning and distributed systems. Developers are not only tasked with selecting the right model for their application, but also trying to understand future operating costs and required application capabilities at scale. Integrating LLM capabilities presents a host of challenges for application developers:
_______________ Introducing Aviary Using Aviary, application developers can rapidly test, deploy, manage and scale one or more pre-configured open source LLMs that work out of the box, while maximizing GPU utilization and reducing cloud costs. With Aviary, Anyscale is democratizing LLM technology and putting it in the hands of any application developer who needs it, whether they work at a small startup or a large enterprise. Optimized for Production LLM Deployment Aviary offers a unified framework to deploy multiple LLMs quickly and add new models in minutes, and is designed for multi-LLM orchestration. Aviary enables continuous testing, allowing developers to A/B test models over time for the best functionality, performance, and cost, delivering the best experience for the end user. Aviary also aids developers on the journey to production deployment of LLMs. It includes libraries, tooling, examples, documentation, and sample code - all available in open source and readily adaptable for small experiments or large evaluations. “Open source models and infrastructure are a great breakthrough for democratizing LLMs,” said Clem Delangue, CEO of Hugging Face. “We’ve attracted an enormous community of open source model builders at Hugging Face. Anything that makes it easier for them to develop and deploy is a win, especially something like Aviary coming from the team behind Ray.” Developers can check out Aviary at https://aviary.anyscale.com/. To join the Ray open source Community please go to Ray.io. About Anyscale Contact and press inquiries: [email protected] 
|
