Considerations around data governance
Data governance refers to aligning all aspects of data strategy, business strategy, and compliance requirements. A three-pronged approach of people, policy, and process will provide oversight for all data operations from the time data touches a system to the point it leaves. Roles and responsibilities dictate who has access to what data, something that needs to be enforced and monitored. Data lineage is tracked to provide accountability for how data has been transformed at various steps. Delta's history functionality provides a good audit trail. A central catalog builds on top of it and provides a central place for defining the rules, enforcing them, and monitoring compliance via audit logs. Some of these catalogs have to be built and stitched together unless a managed platform that has taken care of these aspects is leveraged.
People using data need to be assured of its quality, so being able to define constraints, note when they have...
