Pachyderm has been acquired by Hewlett Packard Enterprise - Learn more

What Is Lineage?

« Back to Glossary Index

When referring to data, the term lineage pertains to the data’s life cycle or line of descent. Data lineage describes the route of data from source to destination and its transformations along the way. It enables businesses to observe the following:

  • The source of data or where it was extracted from
  • The changes or modifications done on the data at each point, including who changed it, when, and why
  • The places where data is permanently or temporarily loaded
  • The integrations with software, applications, or tools

Once you’ve answered “What does lineage mean?” your next question will likely be, why is it important?

 

Why Data Lineage Matters

Organizations must inspect the data journey, even at granular levels, throughout the lineage to ensure that their data came from a trusted source, has been transformed appropriately, and loaded into the correct location.

With lineage, organizations can:

  • Evaluate the trustworthiness of data
  • Identify and correct errors and anomalies
  • Implement changes in processes with lower risk
  • Address incorrect assumptions on the data that skew analysis
  • Protect the flow of data from one destination to another to prevent tampering and leaking of sensitive information
  • Avoid data duplication to streamline operations and lower costs
  • Provide audit trails for regulatory compliance and data governance

Data lineage brings the following benefits to any business:

  • Make Strategic Business Decisions: Armed with high-quality data, companies can be more confident in implementing strategic initiatives for improving performance and overall growth.
  • Improve Regulation Compliance: Today’s businesses face numerous regulatory requirements on data governance. Companies can provide accurate information with data lineage, reducing the risks and costs of non-compliance or wrong data.   
  • Reduce Costs: Many companies collect large volumes of data. But if they can’t manage it effectively, it can result in lost opportunities, data breaches, and errors.   

 

Data Lineage & Pachyderm

Have full visibility of your data throughout its lifecycle with Pachyderm. We offer one of the best-in-class data lineage features, allowing your team to focus on developing better machine learning models. Register for a free 21-day trial to discover how Pachyderm can help.

« Back to Glossary Index