Blockchain for Enterprise: Build scalable blockchain applications with privacy, interoperability, and permissioned features

Byzantine Fault Tolerance (BFT) is a characteristic of a decentralized system that indicates that it can tolerate Byzantine failures. A crash failure is when nodes just stopping to do anything (no messages at all) and Byzantine failure is when nodes just don't do anything or exhibit arbitrary behavior. Basically, Byzantine failures include crash failures.

In any decentralized computing environment where a blockchain data structure is used, there is a risk that one or more rogue or unreliable actors could be a reason for the environment to disband. A server cluster will not work well if a few servers within it lose out on passing data to other servers in a consistent manner. In order to be reliable, the decentralized computing environment has to be designed in a way that it has solutions to these kinds of Byzantine failures.

On blockchain-based decentralised applications, there is, by definition, no central authority, so a special kind of protocol called the consensus protocol is used to achieve BFT. 

In simple terms, you must be wondering how to ensure that everyone has the same copy of the blockchain, and how to know which blockchain is correct when two nodes publish different blockchains? Also, how do you decide who creates the blocks, as there is nothing such as a master node in decentralized architecture? Well,  consensus protocols provide an answer to these questions. A few examples of consensus protocols are Proof-of-Work (PoW), Proof-of-Stake (PoS), Proof-of-Authority (PoA), PBFT, and so on. 

A consensus protocol is designed specially for permissioned or public blockchains. A consensus protocol made for a public blockchain is likely to create security and performance issues when implemented in a permissioned blockchain. Every consensus protocol has different performance and scalability vectors. You have to be alert while selecting a consensus protocol for your blockchain-based DApp. 

You might have come across the term PoA. PoA is a categorisation of consensus protocols in which there is a set of authorities—nodes that are explicitly allowed to create new blocks and secure the blockchain. Ripple's iterative process, PBFT, Clique, Aura, and so on, are examples of PoA-based consensus protocols. 

In blockchain-based applications, user accounts are identified and authenticated using asymmetric key pairs. The private key is used to sign transactions on behalf of the user. Username and password-based accounts systems will not work in blockchain as it cannot be used to prove which user has sent a transaction. The demerits in using private-public key pair include that they are not user-friendly and if you lose the private key then there is no way to recover it. So, it adds a new responsibility for the users to secure their private key. The address of a user account acts as the account identifier on blockchain. The address of a user account is derived from the public key.

Some blockchain applications use the UTXO model for transactions. Blockchain applications such as Bitcoin and MultiChain use this model. Even DLTs such as R3 Corda also use this model. Let's understand this model by understanding how Bitcoin transactions work.

In Bitcoin, a transactions is a collection of zero or more and outputs. These input and output objects are called Unspent Transaction Outputs (UTXO). Outputs of transactions are used as inputs of future transactions. A UTXO can be used as input only once. Each UTXO in Bitcoin contains a denomination and an owner (a Bitcoin address). In this model, the balances of addresses in the unconsumed UTXOs are stored. For a transaction to be valid, these requirements should be met:

A user's balance is computed as the total sum of the denominations of UTXOs that they own. A transaction can consume zero or more UTXOs and produce zero or more UTXOs. For a miner to pay reward to itself, it includes a transaction in the block that consumes zero UTXOs but produces one UTXO with the denomination assigned the amount of Bitcoin it is supposed to award itself. 

A UTXO transaction model is suitable when blockchain transactions involve the transfer of asset, but for non-assets transfer transactions such as recording facts, invoking smart contracts, and so on, this model it not suitable. 

Ethereum is the most popular DApp after Bitcoin. Ethereum is a decentralized platform that allows us to build other blockchain-based DApps on top of it. In Ethereum, we build DApps using Ethereum smart contracts. Smart contracts are applications that run exactly as programmed without any possibility of downtime, censorship, fraud, or third-party interference. Ethereum can be thought of as a platform to deploy and run smart contracts. Ethereum supports two consensus protocols, PoW and PoA (Clique). 

The main public Ethereum network uses PoW for consensus. If you want to deploy your own private Ethereum network, then you have to use PoA. PoW requires a lot of computation power to keep the blockchain secure, therefore it's good for public blockchain use, whereas PoA doesn't have any such computation power requirement; instead it requires a few authority nodes in the network to achieve consensus. 

Quorum is a decentralized platform that allows us to build permissioned blockchain-based DApps on top of it. Actually, Quorum is a fork of Ethereum (actually Quorum is a fork of Go Ethereum, which is an implementation of Ethereum using Golang), therefore if you have ever worked on Ethereum then you will find it easy to learn and build permissioned blockchains using Quorum. Many enterprises select Quorum for building blockchains because of Ethereum's large community, which makes it easy to find Ethereum developers. What makes Quorum different from Ethereum is that it supports privacy (it lets parties do transactions privately); peer whitelisting, so you can mention a list of other nodes that are allowed to connect to your node (in Ethereum this needs to be done at network level); many different flavors of consensus protocols suitable for permissioned blockchain, and provides very high performance. 

Quorum currently supports three consensus protocols, QuorumChain, IBFT, and Raft. We will skip QuorumChain in this book, as Raft and IBFT fulfil all our requirements.

Microsoft Azure provides BaaS to easily build your own Quorum network on the Cloud. But, in this book, we will learn how to install it manually, and we won't be using BaaS. 

Popular node software for Ethereum include Go Ethereum, Ethereum C++, and Parity. Parity also supports two other consensus protocols, other than Ethereum's PoW, which are specifically designed for permissioned blockchains. These consensus protocols are Aura and Tendermint. Many Ethereum developers use parity compared to Quorum when they don't need the extra features provided by Quorum. 

As parity doesn't provide any unique features compared to Quorum, we will be skipping parity in this book. But, once you finish this book, you will find it really easy to grasp parity's concepts and will be able to build something using it too.

MultiChain is a platform to build permissioned blockchain-based DApps. Unique features of MultiChain include permissions management, data streams, and assets. It doesn't support smart contracts. This is an example of a non-smart contract-based platform for building blockchain-based DApps. MultiChain uses round robin validation consensus. 

Initially MultiChain was based on the idea of managing ownership and transfer of assets on blockchain. Operations on assets includes issuance, reissuance, transfer, atomic exchange, escrow, and destruction of assets. Later on, data streams were introduced to provide a different flavor of representing data in MultiChain. Any number of streams can be created in a MultiChain, and each stream acts as an independent append-only collection of items. Operations on streams include creating streams, writing, subscribing, indexing, and retrieving. So, basically, a blockchain use case on MultiChain can be built on a foundation of assets or streams. Finally, permission management is used to control who can connect, transact, create assets/streams, mine/validate, and administrate. 

MultiChain provides maximal compatibility with the Bitcoin ecosystem, including the peer-to-peer protocol, transaction/block formats, the UTXO model, and Bitcoin Core APIs/runtime parameters. So, before you start learning MultiChain, it's better to learn how Bitcoin works at a high level at least.

Before we get into what Hyperledger Fabric 1.0 is, we need to understand what Hyperledger is specifically. Hyperledger is an umbrella project of open source blockchains and related tools, started in December 2015 by the Linux Foundation. At the time of writing this book, there are four projects under Hyperledger:Fabric, Sawtooth, Iroha, and Burrow.

Hyperledger Fabric is the most popular project under Hyperledger. IBM is the main contributer to the project. IBM's Bluemix also provides BaaS to build your own Fabric network on the Cloud easily. 

Hyperledger Fabric 1.0 is a platform to build your own permissioned blockchain-based applications. Currently, at the time of writing this book, Hyperledger Fabric 1.0 supports only distributed architecture, and for the creation of blocks it depends on a central trusted node called the orderer. It supports smart contracts, network permissioning, privacy, and other features. In HLF 1.0, there is a special kind of node called as OSN, which is hosted by a trusted party. This OSN creates blocks and distributes to peers in networks. As you trust this node, there is no need for consensus. HLD 1.0 currently supports CouchDB and LevelDB to store the state of the blockchain. Peers in the network store the state of the blockchain in the LevelDB database, by default. 

HLF 1.0 has a concept of channels to achieve privacy. A channel is a sub-blockchain in the network and allows certain parties to be part of a channel depending on configuration. Actually, every transaction has to belong to a channel and when the HLF 1.0 network is deployed, a default channel is created. OSN can see all the data in all the channels, therefore it should a trusted party. Technically, it's possible to configure the network to have multiple OSNs hosting different channels if you cannot trust a single party for all channels. Even if the traffic is going to be huge or OSN availability is critical, then you can plug Kafka into OSN for better performance and increased stability. We can even have multiple OSNs per channel connected via Kafka if high availability is required. 

Fabric 1.0 has a feature called transaction endorsement, which provides a mechanism of taking approvals from certain parties before sending a transaction. When we say that a transaction has been endorsed by a member in the network, we mean that the member has verified the transaction. Every chaincode (smart contracts in HLF) has an endorsement policy defined to it at the time of deployment. The policy states which members has to endorse the transactions associated with this chaincode. The default policy states that any one member of the channel has to sign the transaction. But, we can define custom policies containing AND and OR operators.

Also, peers of the same channel broadcast blocks to each other regardless of the presence or absence of OSN, but in the absence of OSN new blocks cannot be created for the channel. Peers broadcast blocks using a special protocol called as gossip data dissemination protocol. 

HLF 1.0 has very advanced membership features to control network membership, and that are also internal to a specific organization. In HLF 1.0, you can write chaincodes in Java or Go programming languages. In the future, Fabric 1.0 will come with the Simple Byzantine Fault Tolerance(SBFT) consensus protocol and some other features that will enable us to build DApps. Similarly, there are various new features that are under development and will be released in future as a sub-version of the product. 

BigchainDB is a decentralized database that uses blockchain. BigchainDB is highly scalable and customizable. It uses the Blockchain data structure. It supports features such as rich permissioning, petabytes capacity, advanced querying, linear scaling, and so on. At the time of writing this book, BigchainDB is not production-ready but can be used for building Proof of Concepts (PoCs). We will learn how it works, and will create a basic PoC using it, in later chapters.

InterPlanetary File System (IPFS) is a decentralized filesystem. IPFS uses Distributed Hash Table (DHT) and Merkle Direct Acyclic Graph (DAG) data structures. It uses a protocol similar to Torrent to decide how to move the data around the network. One of the advanced features of IPFS is that it supports file versioning. To achieve file versioning, it uses data structures similar to Git.

Although it's called as a decentralized filesystem, it doesn't adhere to a major property of a filesystem, namely, when we store something in filesystem, it should be there until deleted. But, IPFS doesn't work this way. Each node doesn't store all files, instead it stores only those files it needs. Therefore, if a file is not popular, then many nodes will not have the file therefore there is a huge chance of the file disappearing in the network. Due to this, we can call IPFS a decentralised peer-to-peer file-sharing application. We will learn about about how it works in later chapters.

Corda is a platform on which to build your own permissioned DLT-based applications. Corda is a product of R3. R3 is an enterprise software firm working with over 100 banks, financial institutions, regulators, trade associations, professional services firms, and technology companies to develop Corda. The latest version of Corda is 1.0, which aims to replace legacy softwares used for financial transactions, and enables organisations to digitalize various business process that were cumbersome using legacy software systems:

The preceding diagram shows the high level architecture of a Corda network. Let's understand Corda's architecture at a high level. The idea of R3's Corda is to provide a shared trusted ledger for financial transactions. R3's Corda is not a blockchain platform, therefore there is no concept of blocks, global broadcasts, and so on. All the transactions are point to point. Corda applications are not decentralized. In Corda, smart contracts are called as CorDapps and they are written in either Java or Kotlin. 

Infrastructure services form the nodes in the network that should be hosted by the trusted parties. Network Map publishes IP addresses of all the other nodes, so that nodes can reach out to other nodes. Permissioning service gives permission to nodes to join the network ; the node will receive a root-authority-signed TLS certificate from the network's permissioning service if permitted to join the network. Notaries provide transaction ordering and time stamping services (optionally, a notary also acts as the timestamping authority, verifying that a transaction occurred during a specific time-window before notarizing it). A notary service may be a single network node, a cluster of mutually-trusting nodes, or a cluster of mutually-distrusting nodes.

Notaries are expected to be hosted by enterprises that the network doesn't trust, therefore consensus is required between the notaries, due to which Corda provides various pluggable consensus protocols, such as Raft, BFT, and so on. 

Sometimes, Corda applications need to depend on external application APIs. For example, a multi-currency bank-to-bank payment application built using Corda will need to fetch the exchange rate. In this scenario, the node initiating the transaction can fetch the exchange rate and put on the transaction, but how can you trust that node? Also, every node cannot simply re-fetch the exchange rate to verify if it's correct because by the time other nodes fetch it the rate might have changed, and also this is not a scalable solution. Therefore, Corda provides oracles to solve this issue. There can be one or more oracles in the network. An Oracle is a service that acts as a bridge for communication between two applications. In Corda, the transaction initiator can fetch the information from outside the Corda network and get the information signed from Oraclize to prove its validity. Optionally, Oraclize can also provide the information to the transaction initiator on request. Obviously, the Oraclize should be hosted by trusted parties with respect to what information they provide and sign.

Corda supports any pluggable RDBMS (currently, it is using the H2 database) to store smart contracts data. Data privacy is maintained as to which nodes can see the transactions. Multisignature support is also given by the framework, which enables multiple nodes to sign a transaction. One of the major downsides of Corda is that as there is no global broadcasting, each node has to maintain its own backup and failover redundancy in a traditional way as there is no redundancy built into the network. A node will store transactions and retry sending the messages to the recipient until the recipient has successfully received it. Once the messages are received, the sender has no more responsibility.  

Sawtooth is a decentralized platform to build your own permissioned DApps. The main contributer to Sawtooth is Intel. What makes Sawtooth special is that it uses a Trusted Execution Environment (TEE) (it currently supports Intel's SGX only) for consensus, which makes the network very safe and trustworthy and increases trust in the final result of the consensus. 

Proof of Elapsed Time (PoET) is the name of the consensus protocol that Sawtooth uses. In PoET, there are special types of nodes called as validators. Validators must run their node on an SGX-supported CPU. This is how PoET works.

Every validator requests a wait time from an enclave (a trusted function). The validator with the shortest wait time for a particular transaction block is elected the leader. One function, say CreateTimer, creates a timer for a transaction block that is guaranteed to have been created by the enclave. Another function, say CheckTimer, verifies that the timer was created by the enclave and, if it has expired, creates an attestation that can be used to verify that the validator did, in fact, wait the allotted time before claiming the leadership role. PoET randomly distributes leadership election across the entire population of validators. The probability of election is proportional to the resources contributed (in this case, resources are general-purpose processors with a TEE). An attestation of execution provides information for verifying that the certificate was created within the enclave (and that the validator waited the allotted time). Further, the low cost of participation increases the likelihood that the population of validators will be large, increasing the robustness of the consensus algorithm.

Sawtooth also supports smart contracts (specifically, Ethereum smart contracts can be executed on Sawtooth). Performance-wise, Sawtooth scales well in terms of large numbers of transactions and nodes.

Everledger is a digital registry for diamonds powered by blockchain. It's an example of supply chain management on blockchain. Blockchain was used because, in blockchain, records are immutable. Everledger uses more than 40 features, including color and clarity, to create a diamond's ID. When this information is placed on blockchain, this information becomes a certificate chronicling the jewel's ownership, from mine to ring. Everledger has digitized more than a million diamonds and partnered with firms including Barclays. Participants in the blockchain network, such as merchants, banks, and insurers, can verify if a diamond is legitimate. Everledger is built on the Hyperledger Fabric platform. In the future, they are also planning to add other precious goods to their blockchain.

Let's take an example scenerio and see how blockchain helps in this use case. Alice purchases a diamond, insures it, and registers it on the Everledger blockchain. Next, she loses the diamond and reports it as stolen. The insurance company then compensates her for the loss. Finally, Bob the thief attempts to sell the stolen diamond to Eve the jeweller. She requests verification from Everledger and finds out that it's a stolen diamond. The insurance company is notified about the stolen diamond and they take possession of it.

Walmart's food tracking use case is a combination of blockchain and IoT to make a food product's history transparent and traceable to it's origin. It's an example of supply chain management on blockchain. Walmart's food tracking supply chain management is built on top of the Hyperledger Fabric platform.  

A lot of people die every year due to food poisoning. As soon as someone falls sick or dies due to food poisoning the authorities try to track the source of the food and make sure that all the food items from the source that are distributed is suspended from selling and is called back. This saves lives of a lot of people. But the issue is that as every participation in the supply chain have their own ways and processes storing and retrieving information therefore it takes weeks for the authorities to track the source prevent everyone in the chain from selling the food items. Blockchain in combination with the IoT might just be able to solve this problem.

With every party in the supply chain storing and retrieving information, blockchain can fasten the process of finding the source of a food item. The following list shows additional benefits blockchain can add:

IoT technology, such as sensors and RFID tags, enables real-time data to be written on the blockchain as food products pass along the supply chain.

Let's see an example of what the blockchain records in this case, and who the participants are. The participants are farms from where the food originates, factories where they are packed and processed, cargo companies who ship the food, Walmart stores, and so on. The data recorded on the blockchain is farm origin data, the batch number, factory and processing data, expiration dates, storage temperatures, and shipping details. 

BenBen is a team of research and development engineers dedicated to building innovative products to improve government technology in Ghana. They developed a digital land registry solution using blockchain for Ghana citizens.

In Ghana, banks don't accept land as a collateral when giving loans. That's because in Ghana, a paper registry system is unenforceable in court. This is preventing millions of people from getting loans. 

BenBen provides a top-of-stack land registry and verification platform for financial institutions. This platform captures transactions and verifies the data. BenBen works with financial institutions to update current registries, enable smart transactions, and distribute private keys for clients, to allow automated and trusted property transactions between all parties.

Dubai's housing rental use case is a blockchain application that let's individual expats lease an apartment or renew their housing tenancy contract online within minutes. In Dubai, if an individual wants to take an apartment for rent, then they have to provide KYC documents, cheques as a contract-term guarantee, and create an Ejari (a government contract to legalize the otherwise unpleasant relationship between landlords and tenants in Dubai). In Dubai, most real estate companies rent apartments only if you want to stay for a longer period of time (for instance, at least a year) and to make sure you obey the contract, they ask you to provide postdated cheques as a guarantee, as in Dubai, a cheque bounce is considered a criminal offence. As the process of renting an apartment and renewing the tenancy contract is a cumbersome process for both tenants and real estate companies, Dubai Smart Government (DSG) (a technology arm of Smart Dubai, a city-wide initiative to transform Dubai technologically) launched a mission to make this whole process easier and quicker using blockchain. 

This housing rental application was built using Hyperledger Fabric 1.0, and initially seven entities participated in the network. DSG, General Directorate of Residency and Foreigners Affairs Dubai (DNRD), wasl, Dubai Land Department, The Dubai Electricity and Water Authority (DEWA), Emirates National Bank of Dubai (NBD), and Emirates Islamic (EI) bank were the entities who shared their data on blockchain to make the tenancy contract creation and renewing easier. 

Earlier, DSG and the Emirates Identity Authority (EIDA) launched DubaiID, which allowed Dubai residents a unified access to all eServices provided by government agencies through one login, and interaction with them via the internet. In this blockchain use case, the tenant had to log in to the real estate's portal using DubaiID; in this case, wasl's tenant must have a DubaiID to login. Once logged in, SDG will write the Emirates ID number into blockchain, and DNRD shares visa and passport information on the blockchain for that tenant. Then, wasl's portal redirects users to submit digital cheques using an Emirates NBD or EI bank account. Once digital cheques are submitted, a request is made to DLD via blockchain to renew or create an Ejari. Finally, once Ejari processing is done, DEWA is notified to activate the water and electricity supply. So basically, the first pilot was for individuals who wanted to lease or renew a wasl apartment and had a bank account with Emirates NBD or EI. Soon, more banks and real estate companies will be added to the network to provide this service for more people in Dubai. In this process, it was ensured that a piece of information can only be seen by the concerned parties. 

This use case fits in well as a blockchain use case because a signed immutable ledger was required to store KYC, cheques, and Ejaris and the latter can be proved if the customer or any entity tries to commit fraud. For example, when Emirates NBD issues cheques, if they do it without blockchain and simply make point-to-point API calls, then there is a very good possibility of intentional and unintentional disagreement between ENBD, the tenant, and wasl regarding the existence of a digital cheque or its current status. Therefore, blockchain can be the final tool for reference if any dispute occurs. 

Project Ubin is a digital cash-on-ledger project run in partnership between Monetary Authority of Singapore (MAS) and R3, with the participation of Bank of America (BOA) Merrill Lynch, Credit Suisse, DBS bank, The Hongkong and Shanghai Banking Corporation Limited, J.P. Morgan, Mitsubishi UFJ Financial Groupb (MUFG), OCBC bank, Singapore Exchange (SGX), and United Overseas Bank (UOB), as well as BCS Information Systems as a technology provider.

The aim of Project Ubin is building a digitalized form of the SGD (Singapore's national currency) on a distributed ledger to bring many benefits to Singapore's financial ecosystem. The benefits would be the same as that of any other cryptocurrencies. 

Currently, this application is built using Quorum, but in future it may move to Corda as R3 is one of the partners.

MAS is Singapore's central bank and financial regulatory authority. MAS acts as a settlement agent, operator, and overseer of payment, clearing, and settlement systems in Singapore that focus on safety and efficiency.