The MLOps Ecosystem

When it comes to MLOps, no one tool can do it all. It takes an array of best-of-breed tools to truly automate the entire ML lifecycle. That’s why we’re proud founding members of the AI Infrastructure Alliance (AIIA) a global organization of more than 50 of the most cutting edge companies in AI/ML, working together to build seamless integration points and clean APIs for machine learning.

The Machine Learning Lifecycle

  • Prepare

    Data, Collection, Preparation, Labeling

  • Experiment

    Exploration, Development, Feature, Engineering

  • Train

    ML Model Training, Evaluation/Selection, Monitoring

  • Deploy

    Model Deployment & Serving, Inference Monitoring

Articles

  • The AI Infrastructure Alliance Wants to Build a ‘Canonical Stack’

    Read Article
  • Band of AI startups launch ‘rebel alliance’ for interoperability

    Read Article

The Machine Learning Tech Stack

The diagram below illustrates where Pachyderm and other AIIA members fit into the ML Tech Stack

Download the PDF
CFTPSSTACKNOTEBOOKSDATA ENGINEERING ORCHESTRATION PIPELINEINGESTION TO DEPLOYMENTDATA ENGINEERINGENGINEDATA INGESTION ANDTRANSFORMATION STAGEEXPERIMENTATIONENGINEEXPERIMENTS ANDTESTINGTRAININGENGINETRAINING AND TUNINGDATA FOUNDATIONVERSIONING DATA LAKE WITH LINEAGE TRACKINGINGESTION TO SERVINGMETADATA STOREEXTERNALDATASOURCESLABELINGMODELSECURITYALERTINGENGINELOGGINGENGINEMACHINE LEARNINGTECH STACKDATA SCIENCE EXPERIMENTATION PIPELINEEXPERIMENT TO SERVINGCODE REPOSUCH AS GITRBACROLE BASEDACCESSCONTROLSYNTHETICDATAGENERATION/AUGMENTATIONUSE CASEDEPENDENTMONITORINGEXPERIMENT TO SERVINGCLOUD OR ON-PREM INFRASTRUCTURE(SUCH AS KUBERNETES/IAAS/OBJECT STORE/OS)DEPLOYMENTENGINEDEPLOY TO PRODUCTIONSERVINGENGINEMAKE PREDICTIONSIN APPSDASHBOARDSMODEL TESTING /VALIDATION ENGINEMODEL REPOFEATURE STOREUSE CASE DEPENDENTPRIMARILY FOR STRUCTURED DATAPachydermClear MLFiddlerTectonSeldon

Integrations

Pachyderm integrates with a wide variety of solutions in the ML Tech Stack, and provides sample code and examples for the following partners.

  • From ingestion to serving

    Deploying Seldon and Pachyderm together let's you pull in data from anywhere, build complex models and push them to production with ease.

    Read Article
    plus
  • Labeled data is great but versioned, labeled data is awesome.

    Now you can label all the data in your machine learning pipelines while keeping immutable copies as your labels continually change.

    Watch Video
    plus
  • Experimentation and data engineering unified

    Uniting ClearML's comprehensive experimentation and visualization with Pachyderm's data orchestration engine delivers a complete ML experience.

    Read Article
    plus

Other Partners

Stories from the AIIA Ecosystem

  • The AI Infrastructure Alliance and the Evolution of the Canonical Stack for Machine Learning

    Read Article
  • AI/ML startups align to build a canonical stack and compete with the incumbents

    Read Article
  • The Future of ML and AI Infrastructure and Ethics with Dan Jeffries (Pachyderm, AI Infrastructure Alliance)

    Read Article
  • Seldon and Pachyderm: Two Foundational Pieces of the Machine Learning Loop

    Read Article
  • Fiddler on AIIA

    Read Article
  • Time is right for AI Infrastructure Alliance to Define Better Rules

    Read Article
  • Spell ML on AIIA

    Read Article
  • TerminusDB on AIIA

    Read Article
  • Bloomberg Press release

    Read Article