Building a Data Mesh Strategy

A data mesh may not be helpful to everybody, and adopting it as hype could be overkill. This chapter will discuss the conditions of when a data mesh is applicable and, for those who can benefit from a data mesh architecture, what should be considered before adopting it. In order to build a data mesh, a company needs to first recognize the current state of its analytical solutions and define its future state. This chapter will walk through the main strategic areas to consider when building your data analytics strategy.

In this chapter, we’re going to cover the following main topics:

• Is a data mesh for everybody?
• Aligning your analytics strategy with your business strategy
• Understanding data maturity models
• Building the technology stack
• The analytics team
• Data governance
• Approaches to building your data mesh

Is a data mesh for everybody?

The answer is no. So, who should adopt a data mesh architecture?

• Medium-size companies that have autonomous departments (sales, marketing, finance, human resources) that have their own analytical needs but are forced to centralize data to a central location
• Large multi-national companies that have business across multiple geographical zones and run as independent businesses catering to local market needs
• Small companies and start-ups forecasting exponential growth that rely on data for their business

Which companies do not need a data mesh architecture?

• Small companies that don’t see exponential growth in data should continue using a central data lake or data warehouse.
• Companies that by design or by regulation are prohibited from sharing data across intra-business boundaries will see benefits from some characteristics of a data mesh, but not all. For example, pharma companies working with highly sensitive patient...

Aligning your analytics strategy with your business strategy

A successful data strategy is one that aligns with the business strategy, delivering business outcomes. Depending on the nature of the business and the industry it operates in, there can be different business strategies. A business operating in a very competitive space might want to have a pricing advantage, and hence reducing manufacturing or service costs might be the core strategy for the business. An online business might have a strategy around engaging its customers or marketing the right products to the right audience. It’s important to ensure that the results of your data analytics are providing the right key performance indicators (KPIs) and answering the required questions for your business to align with this strategy. Because, let’s face it, any technology initiative will only get buy-in when it supports the goals of the company.

Understanding and aligning your technology strategy with your business...

Understanding data maturity models

Data analytics maturity models help assess how an organization is leveraging and can leverage data to help the business make decisions. Studies have shown that organizations broadly fall into the following four stages of maturity when it comes to data analytics:

Figure 2.3 – Stages of analytics maturity

Let us learn more about these stages in the following sub-sections.

Stage 1

In this stage, companies ingest data from source systems and transform and move it to a staging area. From the staging area, it is modeled into data marts and served as OLAP cubes. Reporting applications read these cubes and present the data. This stage only caters to structured tabular data available in transactional and legacy sources. The pipelines are all centralized and managed. The system can quickly adapt to new sources as long as those sources provide tabular structured data:

Figure 2.4 – Data...

Building the technology stack

Once you have the business strategy aligned with your data analytics strategy, you can start thinking about the technology stack you will need. Instead of going with the latest and the coolest technology available, you need to align your data analytics strategy with your technology stack. You need to look at your current data maturity and plan your target maturity model based on the analytics required now and in the future. Think about the life cycle of the data as it enters your organization right up to the output it generates – ingestion, integration, transformation, processing, presentation, and archiving.

One of the challenges of modern-day analytics is that data now comes in different formats having different processing needs and processing speeds. Big data has semi-structured data processed on parallel Spark nodes. Transactional data needs to be transformed into OLAP cubes and processed by a massively parallel processing (MPP) data warehouse...

The analytics team

An agile collaborative team is one of the most critical parts of a data analytics strategy for a company. The structure of your team will play a larger role than technology in deciding the efficiency of the analytical framework. The structure of your team will be dependent on the operating model that you build. And, depending on your future data strategy, you might have to regroup teams and even create and hire new roles.

With the growing importance of data governance, a new role called chief data officer (CDO) is being introduced in many companies. The CDO manages the data analytics team along with the new data governance team. A CDO and their team are responsible for the governance and utilization of enterprise-wide data, along with identifying new innovative opportunities to utilize data. Figure 2.8 shows an example of such an organizational structure with a CDO position:

Figure 2.8 – A sample data organization structure

...

Data governance

Finally, the most important part of a modern data strategy is governance. Building a governance plan and documenting it can be a daunting task. It needs to be driven at the leadership level. A good place to start is to begin defining your business glossary and your data classification tags. These two alone can cover a huge portion of your data governance needs.

A business glossary is a standardized understanding of business terms that all employees can refer to and ensure that they are all talking in the same language. It helps remove ambiguity. By associating business glossary terms with data components, those consuming the data are able to understand the content. The term average order value (AOV) might mean different things to different people. But if you have a business glossary that defines what AOV means, then everyone can refer to it and have a common understanding.

Data classification is the process of classifying data into different sensitivity labels...

Approaches to building your data mesh

Depending on the current stage of the analytics system, you could choose from two broad approaches to building your data mesh – a green-field approach or a surround approach.

If your business is at Stage 1 or 2 of data analytics maturity and you currently don’t have much data but see potential growth coming in the future, use the green-field approach where you start with a clean slate, describe your domains and products, and link them together in a data mesh. You then migrate all existing data from your current analytical system into this new data mesh, distributing the data to the domains and the products it belongs to:

Figure 2.10 – Green-field data mesh implementation

If your business is at Stage 3 or Stage 4 of data maturity, then it will be difficult to start from scratch. Many stakeholders might be dependent on the analytics produced by the central analytics system. In this case, you should...

Summary

In this chapter, we saw various elements of a data mesh strategy. We covered cultural, organizational, and technical changes that will need to be adopted to build a solid data strategy. Building a comprehensive, forward-looking data strategy will prove critical to building a collaborative data analytics system using a data mesh architecture. It is very important that a company spends a good amount of time and resources on building this strategy before moving forward with the implementation. Moving from a centralized data analytics structure to a decentralized, collaborative structure is a cultural change. Many companies employ a change management process by seeking help from external consulting companies to help employees adopt the change. Understand your current data maturity and select the appropriate strategy to implement a data mesh.

In the next chapter, we will understand how a data mesh architecture can be deployed using Microsoft Azure. Microsoft has built multiple...