OctoML Unveils New Platform to Deliver DevOps Agility to Machine Learning Deployment
New release automatically generates and delivers trained models as portable software functions, on any hardware
SEATTLE, June 22, 2022 /PRNewswire/ -- OctoML today released a major platform expansion to accelerate the development of AI-powered applications by eliminating bottlenecks in machine learning deployment. This latest release enables app developers and IT operations teams to transform trained ML models into agile, portable, production-ready software functions that easily integrate with their existing application stacks and DevOps workflows.
One of the biggest challenges in enterprise software development today is building reliable and performant AI-powered applications. The problem is 47 percent of fully trained ML models never reach production, and the rest take an average of 12 weeks to deploy. Model deployment is hindered by dependencies between ML training framework, model type, and required hardware at each stage of the model lifecycle. To break this cycle, users need a way to abstract out complexity, strip away dependencies, and deliver models as production-ready software functions.
"AI has the potential to change the world, but it first needs to become sustainable and accessible," said Luis Ceze, CEO, OctoML. "Today's manual, specialized ML deployment workflows are keeping application developers, DevOps engineers and IT operations teams on the sidelines. Our new solution is enabling them to work with models like the rest of their application stack, using their own DevOps workflows and tools. We aim to do that by giving customers the ability to transform models into performant, portable functions that can run on any hardware."
Models-as-functions can run at high performance anywhere from cloud to edge, remaining stable and consistent even as hardware infrastructure changes. This DevOps-inclusive approach eliminates redundancy by unifying two parallel deployment streams—one for AI and the other for raditional software. It also maximizes the success of the investments that have already been made in model creation and model operations.
The new OctoML platform release enables customers to work with existing tools and teams. Intelligent functions can be leveraged with each user's unique combination of model, development environment, developer tools, CI/CD framework, application stack and cloud—all while meeting cost and performance SLAs.
Key platform expansion features include:
Combining NVIDIA Triton with OctoML enables users to more easily choose, integrate, and deploy Triton-powered inference from any framework on mainstream data center servers.
"NVIDIA Triton is the top choice for AI inference and model deployment for workloads of any size, across all major industries worldwide," said Shankar Chandrasekaran, Product Marketing Manager, NVIDIA. "Its portability, versatility and flexibility make it an ideal companion for the OctoML platform."
"NVIDIA Triton enables users to leverage all major deep learning frameworks and acceleration technologies across both GPUs and CPUs," said Jared Roesch, CTO, OctoML. "The OctoML workflow extends the user value of Triton-based deployments by seamlessly integrating OctoML acceleration technology, allowing you to get the most out of both the serving and model layers."
OctoML is a machine learning deployment platform with a mission to make machine learning more accessible. Its industry-leading technology generates production-ready software functions that easily integrate with an organization's existing application stacks and DevOps workflows. Based in Seattle, Washington, the company's investors include Madrona Venture Group, Amplify Partners, Addition, and Tiger Global. For more information, please visit https://octoml.ai/