Reader small image

You're reading from  Driving Data Quality with Data Contracts

Product typeBook
Published inJun 2023
PublisherPackt
ISBN-139781837635009
Edition1st Edition
Right arrow
Author (1)
Andrew Jones
Andrew Jones
author image
Andrew Jones

Andrew Jones is a principal engineer at GoCardless, one of Europe's leading Fintech's. He has over 15 years experience in the industry, with the first half primarily as a software engineer, before he moved into the data infrastructure and data engineering space. Joining GoCardless as its first data engineer, he led his team to build their data platform from scratch. After initially following a typical data architecture and getting frustrated with facing the same old challenges he'd faced for years, he started thinking there must be a better way, which led to him coining and defining the ideas around data contracts. Andrew is a regular speaker and writer, and he is passionate about helping organizations get maximum value from data.
Read more about Andrew Jones

Right arrow

The schema of a data contract

We’ll start this section by looking at the schema of a data contract, what to put in it, and why. Then we’ll look at how to make these schemas accessible to both data generators and consumers, by storing them in a system (or a registry) that is recognized as the source of truth.

We’ll cover these topics in the following subsections:

  • Defining a schema
  • Using a schema registry as the source of truth

Defining a schema

The schema defines the structure of the data. At a minimum, it will hold the complete list of the fields available and their data type.

The following code block shows an example of a schema that defines a Customer record with fields and their types using Protocol Buffers (https://protobuf.dev), as well as a unique field number, as required by Protocol Buffers:

message Customer {
  string id       = 1;
  string name    ...
lock icon
The rest of the page is locked
Previous PageNext Page
You have been reading a chapter from
Driving Data Quality with Data Contracts
Published in: Jun 2023Publisher: PacktISBN-13: 9781837635009

Author (1)

author image
Andrew Jones

Andrew Jones is a principal engineer at GoCardless, one of Europe's leading Fintech's. He has over 15 years experience in the industry, with the first half primarily as a software engineer, before he moved into the data infrastructure and data engineering space. Joining GoCardless as its first data engineer, he led his team to build their data platform from scratch. After initially following a typical data architecture and getting frustrated with facing the same old challenges he'd faced for years, he started thinking there must be a better way, which led to him coining and defining the ideas around data contracts. Andrew is a regular speaker and writer, and he is passionate about helping organizations get maximum value from data.
Read more about Andrew Jones