Performance & Cost Improvements in Pachyderm 2.4

Pachyderm 2.4 is here! At Pachyderm, we’re looking to empower data engineering teams to create automated pipelines with petabyte scalability. With this new release we have some major performance enhancements – with users hitting up to 40% increase in processing throughput. We’re also excited to announce the GA of our extensions and integrations with JupyterLab. 

Our engineering team covers performance improvements, usability improvements in Console, and improvements such as:

• Distributed metadata processing that allows multiple machines to process shards of metadata
• Faster file download times for medium to small files
• Pachyderm Console GUI Improvements; which surfaces backend metrics for debugging and better visualization
• JupyterLab Mount Extension GA