Snowflake made quite a few announcements at its recent Summit 2023, including Snowpark Container Services. In essence, this allows you to deploy, manage and scale containerized workloads with Snowflake infrastructure. But what does it mean in practice?
We already briefly covered Snowflake Container Services in our Snowflake Summit 2023 overview article. The new service will have to complement Snowpark after its private preview. Snowpark is Snowflake’s developer framework to enable writing code in Python, Java and Scala, for example. Among other things, it allows developers to build data pipelines in their preferred language.
Meanwhile, containers are an industry standard for housing code so that it can run in different environments. “Especially for sophisticated AI/ML models and full-stack data-intensive apps,” Snowflake said. There are often huge amounts of proprietary data involved with these types of applications. “The complexity of creating, developing and running these workloads at scale is generally complex, forcing developers and data scientists to spend more time managing the compute and clusters for these applications rather than focusing on the business problem.”
In addition, Snowflake notes that full-stack apps and large language models (LLMs) can’t run directly where data is governed. That makes managing the security of the data difficult, sometimes almost impossible, according to the company.
The solution
For Snowflake, it was time to make full-stack apps, LLMs and other extended data solutions more secure and closer to the data. It will now do just that with Snowpark Container Services, which is in private preview and will continue to expand over time. Snowflake describes it as an additional Snowpark runtime option for developers. It enables them to “effortlessly deploy, manage and scale containerized workloads (jobs, services, service functions) using a secure Snowflake-managed infrastructure with configurable hardware options, such as GPUs.”
Snowpark Container Services takes away the task of managing and maintaining compute and clusters for containers. Containers run in Snowflake, eliminating the need for data to go out of the environment for an AI model or application. With it, your company can bring internally developed data solutions to the Data Cloud, as well as install and run third-party solutions. The image below summarizes the components involved.
What does it enable in practice?
This approach sounds pretty interesting, although we also wondered what it really enables. Fortunately, during the Summit 2023 user conference, Snowflake was able to share examples of tech partners using Snowpark Container Services. We’ll cover two below that, as far as we’re concerned, show how data engineers/data scientists benefit.
The first is SAS, whose Viya platform is used within organizations for model development. Integrating SAS with Snowflake should ensure that the entire analytics and AI lifecycle, from data discovery and model development to ultimately making decisions based on data, is supported. “Snowpark Container Services enables our joint customers to use SAS Viya AI and decisioning capabilities in Snowflake to answer critical questions, discover insights and make better decisions,” said SAS CTO Bryan Harris. “We are proud to partner with Snowflake, as customers in today’s world need technology companies to collaborate on their behalf to drive greater productivity gains and innovation in their businesses. Together, we’re democratizing the creation and deployment of AI, and empowering our customers to build more efficiently in the Snowflake Data Cloud.”
Another Snowflake tech partner that will use Snowpark Container Services is Alteryx. Alteryx’s platform helps companies automate processes around analytics, machine learning and data science. Through the new integration with Snowflake, Alteryx aims to make it easier to find and transform data. It expands the capabilities of Alteryx’s cloud-connected experiences, as the company itself describes the integration. “With Alteryx’s Snowflake Execution for Desktop, joint customers can leverage Alteryx’s low-code advanced transformation tools and perform data processing for their workflows entirely within Snowflake,” said Torsten Grabs, senior director of product management, Snowflake. “This new capability securely democratizes data access for business users and analysts, unlocking new use cases within customer organizations for both Snowflake and Alteryx.”
Also read: Nvidia lowers case for gen-AI data concerns on Snowflake secure LLMs
18 hours ago
