Pachyderm Enterprise 2 Brings the Data Foundation for Machine Learning to Your Environment


Today we’re announcing the release of Pachyderm Enterprise 2, the latest version of Pachyderm’s user-deployed software. Pachyderm Enterprise 2 gives data science teams a powerful new data layer allowing them to deliver total reproducibility at scale throughout the entire machine learning lifecycle. Data science teams already know Pachyderm’s unique data versioning and data lineage features that let teams track every change to their models, code and data. Pachyderm also shows how each change relates to the others from ingestion to serving. Our latest user-deployable version, Pachyderm Enterprise 2, brings a range of essential new features, like Global Identifiers, a dynamic new web console, and data versioning and pipeline speed improvements, to your environment. In the limited release of Pachyderm 2, we wrote in detail about the improvements we made, and you can read that here. Pachyderm Enterprise 2 also makes deploying and operating those on-premises or cloud clusters easy, through robust new management features, including a new Helm 3 chart.

Reducing MLOps costs for machine learning and data science code means little if the data layer itself raises costs by being hard to manage. Pachyderm Enterprise 2 includes a single point of management for licensing, authentication, access controls, and configuration management for all your Pachyderm workspaces across all your clusters. All these features are deployable and customizable through a new Helm 3 chart that makes standardized, centralized operations easy and cost-effective.

Pachyderm Enterprise 2 includes the new Enterprise Server, which can be deployed integrated with a single cluster or separately to manage a suite of clusters. Typically, organizations will create development, staging and production clusters to simplify MLOps workflows. Implementing consistent best practices across those clusters for licensing, access controls, configuration management, and authentication used to require separate configuration of each cluster and maintenance of those configurations. Pachyderm Enterprise allows those configurations and policies to be stored in the Enterprise Server, and activated with one registration command when a new cluster is spun up. It makes best practices easy and efficient to manage.

The new Helm 3 chart makes customizing, spinning up, and maintaining those clusters easier than ever, using an industry-standard configuration that is easily customizable.

“Many of our enterprise customers run multiple Pachyderm workspaces in different Kubernetes clusters. Pachyderm Enterprise 2 makes that easy to do with our new Helm 3 chart and centralized management server, " says Joe Doliner, CEO of Pachyderm. “Our enterprise customers can gain complete, cost-effective reproducibility at scale in their own environments.”

Fast and flexible pipelines along with automated data versioning and immutable lineage is how Pachyderm helps customers mature their MLOps practices. Pachyderm Enterprise 2 builds on that strength by allowing consistent management and access controls across the enterprise.

About the Author

Joey Zwicker

Joey is Co-Founder and COO of Pachyderm. He is a multitasking wizard and leverages his technical background to be better at the enormous breadth of tasks actually on his plate.