Using Managed Ledger Databases for Finance Data Integrity

In the previous chapter, we discussed how we can use the database-locking capabilities of relational database management systems (RDBMSs) to prevent data integrity issues in financial transactions. In this chapter, we will be shifting our focus a bit and discussing another type of database called ledger databases. Specifically, we’ll use Amazon Quantum Ledger Database (QLDB), which is a purpose-built fully managed ledger database built by the cloud service provider Amazon Web Services (AWS). We will explore the core concepts of this database and learn how its unique set of features and capabilities can help maintain and manage the integrity of financial records and transactions. Of course, we’ll dive deep into various hands-on solutions and examples to help us better understand how we can utilize the features of this database service for financial data integrity management.

The flow of our discussion will be...

Technical requirements

Before we dive in, we must have the following in place:

• A web browser
• Any text editor (such as Notepad or Sublime Text) where we can store the specific values (for example, the digest) that will be used in the hands-on solutions in this chapter

The PartiQL queries (a SQL-compatible query language used in QLDB and certain databases) are saved in the official Packt GitHub repository and can be accessed from https://github.com/PacktPublishing/Managing-Data-Integrity-for-Finance/tree/main/ch09.

Note

In this chapter, we will be setting up a managed ledger database resource on AWS. If this is your first time using AWS, feel free to check out the following video to help you get started: https://www.youtube.com/watch?v=a9__D53WsUs.

Once these are ready, you may proceed.

Introduction to ledger databases

In the previous chapters of this book, we primarily made use of the capabilities of RDBMSs to help us solve various types of data integrity issues and challenges. While this type of database would be a good default option for most finance and technology professionals, other types of purpose-built databases can offer significant advantages in specific use cases. These purpose-built databases may include NoSQL databases, graph databases, time-series databases, and more, each designed to excel in handling particular data models and workloads. One such example that maintains an immutable and verifiable record of transactions over time is the ledger database. This type of database makes use of financial ledger concepts where each journal is appended only and cannot be altered once recorded. Because of these properties, companies from various industries that have strict requirements, which require transparency can make use of this type of database.

In...

Reviewing the internals of ledger databases

Now that the ledger has been created, we will build on what we have discussed and continue covering more important concepts of QLDB. We will start by discussing the digest, creating a table, and inserting records into this table through queries.

Getting the digest

As we mentioned at the beginning of this chapter, a digest is a cryptographic version of your ledger’s complete history at a specific point in time. This is one of the key components for the verification step, which we will discuss in the Understanding how ledger databases prevent data integrity issues section.

Continuing with our example, locate and click on the radio button beside myBanking to activate the Edit, Query ledger, Get digest, and Delete options. When we click on Get digest, it will generate the digest, as seen in Figure 9.20. This will contain the details that we will use later during the verification step:

Figure 9.20 –...

Understanding how ledger databases prevent data integrity issues

In this section, we will discuss the steps and components to perform the various verification procedures. We will dive deep into the queries on how to do this, as well as discuss the immutability of the ledger by covering the built-in history function. By doing this, we will see how QLDB can maintain a complete record of the changes we have made and show how cryptographic verification is done. These features are what make this database unique and help maintain the integrity of the data.

Note

Cryptographic verification in QLDB is a method that ensures the integrity and authenticity of the data within the ledger. This is possible by utilizing hash chaining combined with a Merkle tree-based model to perform the verification. Because of this capability, the integrity of the ledger is maintained, making it well-suited for use where an accurate and reliable record of transactions is essential.

Now that we have covered...

Exploring the best practices when using ledger databases

In this chapter, we covered a lot of important concepts on how QLDB retains a comprehensive and verifiable record of modifications made over time. This capability enhances the ledger’s transparency and maintains its data integrity. In this section, we will look at the best practices when working with ledger databases. They are as follows:

• Data structure: Implement a data structure that fits your needs. Organize and design the tables in a way that makes it easy to search for and retrieve data efficiently.
• Immutability: Since QLDB is immutable and contains a verifiable history of all the changes, make use of this capability to maintain data integrity and traceability.
• Querying: There are some constraints concerning querying the data in QLDB. These must be taken into consideration when writing queries for optimal performance.
• Cost management: Make sure you have a good understanding of how QLDB pricing...

Summary

In this chapter, we discussed how ledger databases can help us with data integrity management. In our hands-on examples, we specifically used Amazon QLDB, a fully managed ledger database available on AWS. We started by creating an AWS account and then creating an S3 bucket where our journal data from QLDB would be exported. After that, we created the QLDB ledger resource and covered the basic concepts such as getting the digest, the PartiQL editor, how to create a table, and inserting data into the table. Then, we covered how verification is done in QLDB, and why being immutable and cryptographically verifiable are crucial in preventing data integrity issues. We also covered how to work with the built-in history function and data even though they have been deleted. Lastly, we covered the best practices when using ledger databases, particularly for QLDB. Now that you are equipped with a solid understanding of these concepts, you know how the use of this type of database can...

Further reading

To dive deeper into the topics we discussed in this chapter, feel free to explore these additional resources:

• What is Amazon QLDB?:

  https://docs.aws.amazon.com/qldb/latest/developerguide/what-is.html

• From relational to ledger:

  https://docs.aws.amazon.com/qldb/latest/developerguide/what-is.relational-ledger.html

• Amazon QLDB glossary:

  https://docs.aws.amazon.com/qldb/latest/developerguide/qldb-glossary.html

• Data verification in Amazon QLDB:

  https://docs.aws.amazon.com/qldb/latest/developerguide/verification.html

• Querying revision history:

  https://docs.aws.amazon.com/qldb/latest/developerguide/working.history.html

• Optimizing query performance:

  https://docs.aws.amazon.com/qldb/latest/developerguide/working.optimize.html

• Amazon Quantum Ledger Database (QLDB) pricing:

  https://aws.amazon.com/qldb/pricing/