How do you implement data lineage and metadata management?

Data lineage tracks how data flows through systems from source to consumption, enabling impact analysis, debugging, and compliance. Types: table-level lineage (which tables feed into which other tables), column-level lineage (how individual columns transform through the pipeline), and business lineage (how business metrics are derived from raw data). Implementation: automated lineage extraction from SQL parsing (track SELECT INTO, INSERT, CREATE TABLE AS patterns), pipeline metadata (Airflow DAGs, dbt models expose dependencies), and API-based tracking (log data movements through application code). Tools: OpenLineage (open standard for lineage metadata), DataHub (LinkedIn's open-source metadata platform), Atlan (data catalog with lineage), and dbt (provides lineage through model references). Metadata management: technical metadata (schema, data types, statistics, freshness), business metadata (descriptions, ownership, classification, glossary terms), operational metadata (pipeline run history, data quality scores, access logs), and social metadata (usage popularity, user ratings, discussions). Data catalog: implement a searchable inventory of all data assets, tag datasets with business domain, sensitivity classification, and quality tier, enable self-service data discovery for analysts and scientists, and track dataset popularity and usage patterns. Impact analysis: before changing a table schema, trace downstream dependencies to assess blast radius, notify affected teams, and coordinate changes. Compliance: lineage supports GDPR (trace where personal data flows for right-to-deletion compliance), SOX (demonstrate data accuracy from source to financial report), and general audit requirements. Implementation approach: start with automated lineage from your transformation tool (dbt, Spark), augment with API tracking for custom applications, and maintain a catalog with manual enrichment of business context.