Data Governance

Data governance can be irresponsibly defined as, everything you need to do to ensure data is compliant, secure, accurate, available, and valuable.

Most data professionals and technologists are rebels. Instinctively, we worry more about making something work rather than reflecting on the role of our solution in the extensive landscape of our organizations. We often enjoy novelty and working with new technology but not the administration and processes to support that work. In fact, we secretly believe most people reading this book will skip this chapter after reading the title. However, we urge you to stay a bit longer. Data governance is not the sexiest topic, but it is what separates mature data teams from the rest.

In short, doing data governance well can be challenging. It requires a lot of discipline, empathy, detail-oriented thinking, and (most often lacking) dedication. But it is probably one of the most essential activities to ensure that your work benefits...

Understanding data governance

Data governance is a comprehensive and disciplined approach to managing and protecting an organization’s data assets. It involves establishing a framework encompassing people, processes, rules, policies, tools, and guidelines to ensure data quality, security, and compliance.

Any data governance framework should specify the interactions between its three pillars: people, process, and technology.

The people pillar involves establishing clear roles and responsibilities for individuals within the organization. This includes people who create guidelines, oversee compliance, and implement these policies in the tools we use (the last is probably you). It is crucial to have a team that understands the importance of data governance and can effectively implement and enforce the necessary policies and procedures.

The process pillar involves developing data lifecycle management processes that cover data creation, storage, usage, sharing, and disposal...

Applying data governance in analytics engineering

So far, the concept of data governance might feel overwhelming or a bit fuzzy. If so, there is no need to worry. Nobody is going to ask you for a textbook definition. In fact, nobody else in your team may have ever heard of governance.

Nevertheless, even if they do not call by its name, your colleagues will often discuss the issues caused by poor governance. They will talk about how much time they waste cleaning data or their frustration when nobody fixes data quality at the source. For this reason, from now on, we will talk about governance in the way that most people understand: highlighting its main topics and demonstrating how they are relevant to your work.

Governance is interwoven into your role as an analytics engineer, especially if enabling data analysts and other data consumers is among your responsibilities. In this section, we will provide a comprehensive overview of the essential elements that data teams need to consider...

Addressing critical areas for seamless data governance

Enabling data governance at your organization is not going to be seamless. There will always be some challenges. From a lack of awareness and understanding to resistance to change and adoption, we will delve into these challenges and provide insights on overcoming them.

Resistance to change and adoption

People’s resistance to change in the field of IT is extremely common, especially if your organization is not primarily or entirely online. Technology moves fast, and people must constantly adapt to new systems, processes, and tools. However, many tend to resist change due to various reasons. Some fear the unknown, some do not have enough broadband to relearn how to do their job, and some just want to maintain the status quo. Regardless of the reason, here are some strategies to address this resistance:

• Effective communication: It is important to communicate the reasons behind the change, its benefits, and how...

Summary

Data governance refers to any task you must do to make your data compliant, secure, accurate, available, and useful. Even though organizations often ignore it, it sets mature data teams apart. It enables you to work towards your strategic goals and reduce the hours wasted maintaining and fixing existing data assets.

In this chapter, we discuss some key topics in governance, such as ownership, data quality, managing data assets, training, and data modeling. A recurrent theme is that building governance roadmaps from scratch is generally not your responsibility. However, analytics engineers are in a privileged position to understand issues with the data and have enough technical knowledge to correct them at the source.

Working on data governance is never going to be easy. You will face resistance to change and need to get buy-in from your stakeholders to ensure the success of your initiatives. However, any goal you achieve will translate into a much better experience for...