Organizations of every size and across every industry are looking to use generative AI to fundamentally transform the business landscape with reimagined customer experiences, increased employee productivity, new levels of creativity, and optimized business processes. A recent study by Telecom Advisory Services, a globally recognized research and consulting firm that specializes in economic impact studies, shows that cloud-enabled AI will add more than $1 trillion to global GDP from 2024 to 2030.
Organizations are looking to accelerate the process of building new AI solutions. They use fully managed services such as Amazon SageMaker AI to build, train and deploy generative AI models. Oftentimes, they also want to integrate their choice of purpose-built AI development tools to build their models on SageMaker AI.
However, the process of identifying appropriate applications is complex and demanding, requiring significant effort to make sure that the selected application meets an organization’s specific business needs. Deploying, upgrading, managing, and scaling the selected application also demands considerable time and effort. To adhere to rigorous security and compliance protocols, organizations also need their data to stay within the confines of their security boundaries without the need to store it in a software as a service (SaaS) provider-owned infrastructure.
This increases the time it takes for customers to go from data to insights. Our customers want a simple and secure way to find the best applications, integrate the selected applications into their machine learning (ML) and generative AI development environment, manage and scale their AI projects.
Introducing Amazon SageMaker partner AI apps
Today, we’re excited to announce that AI apps from AWS Partners are now available in SageMaker. You can now find, deploy, and use these AI apps privately and securely, all without leaving SageMaker AI, so you can develop performant AI models faster.
Industry-leading app providers
The first group of partners and applications—shown in the following figure—that we’re including are Comet and its model experiment tracking application, Deepchecks and its large language model (LLM) quality and evaluation application, Fiddler and its model observability application, and Lakera and its AI security application.
Managed and secure
These applications are fully managed by SageMaker AI, so customers don’t have to worry about provisioning, scaling, and maintaining the underlying infrastructure. SageMaker AI makes sure that sensitive data stays completely within each customer’s SageMaker environment and will never be shared with a third party.
Available in SageMaker AI and SageMaker Unified Studio (preview)
Data scientists and ML engineers can access these applications from Amazon SageMaker AI (formerly known as Amazon SageMaker) and from SageMaker Unified Studio. This capability enables data scientists and ML engineers to seamlessly access the tools they require, enhancing their productivity and accelerating the development and deployment of AI products. It also empowers data scientists and ML engineers to do more with their models by collaborating seamlessly with their colleagues in data and analytics teams.
Seamless workflow integration
Direct integration with SageMaker AI provides a smooth user experience, from model building and deployment to ongoing production monitoring, all within your SageMaker development environment. For example, a data scientist can run experiments in their SageMaker Studio or SageMaker Unified Studio Jupyter notebook and then use the Comet ML app for visualizing and comparing those experiments.
Streamlined access
Use AWS credits to use partner apps without navigating lengthy procurement or approval processes, accelerating adoption and scaling of AI observability.
Application deep dive
The integration of these AI apps within SageMaker Studio enables you to build AI models and solutions without leaving your SageMaker development environment. Let’s take a look at the initial group of apps launched at re:Invent 2024.
Comet
Comet provides an end-to-end model evaluation solution for AI developers with best-in-class tooling for experiment tracking and model production monitoring. Comet has been trusted by enterprise customers and academic teams since 2017. Within SageMaker Studio, Notebooks and Pipelines, data scientists, ML engineers, and AI researchers can use Comet’s robust tracking and monitoring capabilities to oversee model lifecycles from training through production, bringing transparency and reproducibility to ML workflows.
You can access the Comet UI directly from SageMaker Studio and SageMaker Unified Studio without the need to provide additional credentials. The app infrastructure is deployed, managed, and supported by AWS, providing a holistic experience and seamless integration. This means each Comet deployment through SageMaker AI is securely isolated and provisioned automatically. You can seamlessly integrate Comet’s advanced tools without altering their existing your SageMaker AI workflows. To learn more, visit Comet.
Deepchecks
Deepchecks specializes in LLM evaluation. Their validation capabilities include automatic scoring, version comparison, and auto-calculated metrics for properties such as relevance, coverage, and grounded-in-context. These capabilities enable organizations to rigorously test, monitor, and improve their LLM applications while maintaining complete data sovereignty.
Deepchecks’s state-of-the-art automatic scoring capabilities for LLM applications, paired with the infrastructure and purpose-built tools provided by SageMaker AI for each step of the ML and FM lifecycle, makes it possible for AI teams to improve their models’ quality and compliance.
Starting today, organizations using AWS can immediately work with Deepchecks’s LLM evaluation tools in their environment, minimizing security and privacy concerns because data remains fully contained within their AWS environments. This integration also removes the overhead of onboarding a third-party vendor, because legal and procurement aspects are streamlined by AWS. To learn more, visit Deepchecks.
Fiddler AI
The Fiddler AI Observability solution allows data science, engineering, and line-of-business teams to validate, monitor, analyze, and improve ML models deployed on SageMaker AI.
With Fiddler’s advanced capabilities, users can track model performance, monitor for data drift and integrity, and receive alerts for immediate diagnostics and root cause analysis. This proactive approach allows teams to quickly resolve issues, continuously improving model reliability and performance.
Lakera
Lakera partners with enterprises and high-growth technology companies to unlock their generative AI transformation. Lakera’s application Lakera Guard provides real-time visibility, protection, and control for generative AI applications. By protecting sensitive data, mitigating prompt attacks, and creating guardrails, Lakera Guard makes sure that your generative AI always interacts as expected.
Starting today, you can set up a dedicated instance of Lakera Guard within SageMaker AI that ensures data privacy and delivers low-latency performance, with the flexibility to scale alongside your generative AI application’s evolving needs.
See how customers are using partner apps
“The AI/ML team at Natwest Group leverages SageMaker and Comet to rapidly develop customer solutions, from swift fraud detection to in-depth analysis of customer interactions. With Comet now a SageMaker partner app, we streamline our tech and enhance our developers’ workflow, improving experiment tracking and model monitoring. This leads to better results and experiences for our customers.”
– Greig Cowan, Head of AI and Data Science, NatWest Group.
“Amazon SageMaker plays a pivotal role in the development and operation of Ping Identity’s homegrown AI and ML infrastructure. The SageMaker partner AI apps capability will enable us to deliver faster, more effective ML-powered functionality to our customers as a private, fully managed service, supporting our strict security and privacy requirements while reducing operational overhead.”
– Ran Wasserman, Principal Architect, Ping Identity.
Start building with AI apps from AWS partners
Amazon SageMaker AI provides access to a highly curated selection of apps from industry leading providers that are designed and certified to run natively and privately on SageMaker AI. Data scientists and developers can quickly find, deploy, and use these applications within SageMaker AI and the new unified studio to accelerate their ML and generative AI model building journey.
You can access all available SageMaker partner AI apps directly from SageMaker AI and SageMaker Unified Studio. Click through to view a specific app’s functionality, licensing terms, and estimated costs for deployment. After subscribing, you can configure the infrastructure that your app will run on by selecting a deployment tier and additional configuration parameters. After the app finishes the provisioning process, you will be able to assign access to your users, who will find the app ready to use in their SageMaker Studio and SageMaker Unified Studio environments.
About the authors
Gwen Chen is a Senior Generative AI Product Marketing Manager at AWS. She started working on AI products in 2018. Gwen has launched an NLP-powered app building product, MLOps, generative AI-powered assistants for data integration and model building, and inference capabilities. Gwen graduated from a dual master degree program of science and business with Duke and UNC Kenan-Flagler. Gwen likes listening to podcasts, skiing, and dancing.
Naufal Mir is a Senior Generative AI/ML Specialist Solutions Architect at AWS. He focuses on helping customers build, train, deploy, and migrate ML workloads to SageMaker. He previously worked at financial services institutes developing and operating systems at scale. He enjoys ultra-endurance running and cycling.
Kunal Jha is a Senior Product Manager at AWS. He is focused on building Amazon SageMaker Studio as the IDE of choice for all ML development steps. In his spare time, Kunal enjoys skiing, scuba diving and exploring the Pacific Northwest. You can find him on LinkedIn.
Eric Peña is a Senior Technical Product Manager in the AWS Artificial Intelligence Platforms team, working on Amazon SageMaker Interactive Machine Learning. He currently focuses on IDE integrations on SageMaker Studio. He holds an MBA degree from MIT Sloan and outside of work enjoys playing basketball and football.
Arkaprava De is a manager leading the SageMaker Studio Apps team at AWS. He has been at Amazon for over 9 years and is currently working on improving the Amazon SageMaker Studio IDE experience. You can find him on LinkedIn.
Zuoyuan Huang is a Software Development Manager at AWS. He has been at Amazon for over 5 years, and has been focusing on building SageMaker Studio apps and IDE experience. You can find him on LinkedIn.
