Industry Best Practices

Industry best practices are proven ideas, methods, techniques, and guidelines. In the Information Technology (IT) industry, there are many established best practices deemed as successful from lessons learned on what works and doesn’t. These experiences have revealed the best course of action on how to achieve data management at a high scale. A best practice achieves quality, greater productivity, lower cost, and increased profit. Gartner defines best practices as “a group of tasks that optimizes the efficiency (cost and risk) or effectiveness (service level) of the business discipline or process to which it contributes. It must be implementable, replicable, transferable and adaptable across industries.”

In this chapter, we’ll review the top 16 best practices for data management in the IT industry that focus on typical pain points such as data silos, data breaches, and data access bottlenecks. We will revisit many of the best practices...

Top 16 best practices

There are many well-known data management pain points in the IT industry. Some of these pain points are expensive data operations, data silos, bad business decisions from unreliable data, data breaches, and data access bottlenecks. These challenges are faced by enterprises due to the high volume of data and its extensive proliferation. Data is growing at an exponentially high rate each day and it’s difficult for organizations to manage data effectively and efficiently while ensuring data is protected and of high quality. Best practices alongside architecture principles and data management frameworks are embedded into a Data Fabric design in order to address many of these pain points. These attributes create a high-performing and mature data architecture capable of handling a diverse set of use cases, across industries, that is optimized to address typical pain points faced by organizations in data management.

The key best practices discussed throughout...

Data strategy best practices

Here are key best practices to create and execute a value-driven data strategy. Let’s review each one.

Best practice 1

Create a clear and comprehensive data strategy.

A data strategy must align with the business goals and overall framework of how data will be used and managed within an organization. It needs to include standards for how data will be discovered, integrated, accessed, shared, and protected. It needs to address how data will meet regulatory compliance policies, Master Data Management, and data democratization. There needs to be an assurance that both data and metadata have a quality control framework in place to achieve data trust. A data strategy needs to have a clear path on how an organization will accomplish data monetization.

A data strategy is a living document that needs to be continuously updated to align with business goals. It should have a clear maintenance process with frequent reviews and identification of authors...

Data architecture best practices

What are the best practices in designing a quality data architecture? It’s a careful balance between positive trade-offs versus negative trade-offs.

The following is a list of best practices in data architecture.

Best practice 5

Assess and document architecture decisions.

A trade-off analysis should be completed when designing a data architecture that carefully evaluates each option, weighing the pros and cons. Documented architecture decisions that capture the options considered and the rationale of going one way or another should be captured as Architecture Decision Records (ADRs). This offers tribal knowledge in the decision-making process, which needs to cover both business and technical reasoning behind the decisions made. ADRs create better collaboration and understanding for future changes required in a data architecture especially as data roles change within teams.

Why should you care?

Trade-off analysis and ADRs avoid...

Data Integration and Self-Service best practices

Data Integration and Self-Service focus on the development and delivery of data throughout its life cycle, handling inbound and outbound data processing and facilitating data access.

The following is a list of best practices in data processing and Self-Service data access.

Best practice 10

Apply DataOps principles to the development and delivery of data.

DataOps is a best practice framework that accelerates the development of data and quality across its entire life cycle with high efficiency and quality. This is especially important when integrating data across distributed complex systems and environments. Concepts such as quality control, version control, data orchestration, continuous integration/continuous deployment (CI/CD), automated testing, automated deployments, and data monitoring are applied to data and metadata. Feedback loops establish an open communication channel for customers to drive ongoing quality improvements...

Data Governance best practices

Data Governance in today’s era enables access to high-quality and trusted data via automation and mature technologies despite the need to enforce security and regulatory requirements. There is consensus across the IT industry on the criticality of Data Governance to be successful in data management.

Let’s review best practice 13.

Best practice 13

Define and enforce data policies that address Data Privacy, Data Protection, and Data Security to avoid data breaches.

There should be a high degree of consideration of how an organization addresses Data Privacy, Data Protection, and Data Security. In the event of a data breach, there needs to be a planned course of action to avoid further risk. Data policies need to handle Data Privacy, Data Retention, and Data Security. Data should be classified as sensitive, government, financial, or other to drive data protection. Automated enforcement policies should be leveraged to take appropriate...

Summary

In this chapter, we have summarized 16 best practices in data management relating to the IT industry. We have recapped best practices across four categories: Data Strategy, Data Architecture, Data Integration and Self-Service, and Data Governance. Each category represents the top best practices that are baked into a Data Fabric architecture. The goals of Data Fabric architecture are to deliver high-quality, governed, protected, and secure data in a high-scale and frictionless manner. Figure 10.1 depicts a logical Data Fabric architecture covered in Chapter 8, Designing Data Integration and Self-Service.

Figure 10.1 – Logical Data Fabric architecture

This concludes the book. You should now have a deeper understanding of and appreciation for Data Fabric architecture. You can comprehend how it can be used together with other architectures and frameworks, the tremendous value statement it offers, its flexible nature, and its ability to support...