Free eBook - Blockchain By Example

4.5 (4 reviews total)
By Bellaj Badr , Richard Horrocks , Xun (Brian) Wu
  • A new free eBook every day on the latest in tech
  • 30 permanently free eBooks from our core tech library
  1. Say Hello to Blockchain
About this book
The Blockchain is a revolution promising a new world without middlemen. Technically, it is an immutable and tamper-proof distributed ledger of all transactions across a peer-to-peer network. With this book, you will get to grips with the blockchain ecosystem to build real-world projects. This book will walk you through the process of building multiple blockchain projects with different complexity levels and hurdles. Each project will teach you just enough about the field's leading technologies, Bitcoin, Ethereum, Quorum, and Hyperledger in order to be productive from the outset. As you make your way through the chapters, you will cover the major challenges that are associated with blockchain ecosystems such as scalability, integration, and distributed file management. In the concluding chapters, you’ll learn to build blockchain projects for business, run your ICO, and even create your own cryptocurrency. Blockchain by Example also covers a range of projects such as Bitcoin payment systems, supply chains on Hyperledger, and developing a Tontine Bank Every is using Ethereum. By the end of this book, you will not only be able to tackle common issues in the blockchain ecosystem, but also design and build reliable and scalable distributed systems.
Publication date:
November 2018
Publisher
Packt
Pages
528
ISBN
9781788475686

 

Say Hello to Blockchain

What is blockchain? Certainly, with the huge hype around it, you must have heard or come across this questionit might be even the reason why you are reading this book. Let's discover, in this first chapter from a developer's standpoint, what's behind the hype.

As you might know, blockchain is an emerging technology that has the potential to dramatically revolutionize many different fields. This potential is primarily based on its ability to offer people a trustworthy channel to transfer value or real assets (tokenization) over the internet.

Blockchain has the capacity to move us from the internet of information to the internet of value, potentially breaking our existing financial systems.

Blockchain is in many ways a revolution, similar to the internet when it was conceived—certainly not a passing trend. The reason for this is that it presents a solution to a previously unsolved financial dilemma. For the first time in history, we are able to establish trust within trustless environments (such as the internet), without relying on an authority. As a result, some refer to blockchain as a trust machine.

The potential impact of blockchain is hugeit goes far beyond the mere decentralization of the financial sector. In fact, its ability to circumvent intermediaries opens the door to redefine almost every field revolving around technology—even the internetpushing us toward a peer-to-peer world.

Through this short introduction, I am trying to give you a foretaste of the importance of our topic, and to confirm that your choice to learn about the technology is timely. As the book's name suggests, the approach we will be following throughout this book is to build concrete blockchain projects, instead of laying out abstract concepts.

Nonetheless, in spite of its less technical nature, the prime objective of this introductory chapter is to provide you with the necessary background to build the various projects presented in this book.

In this chapter, we will cover the following topics:

  • What cryptocurrency is
  • What blockchain is
  • How to send and receive bitcoins
  • How to store data into a bitcoin blockchain using JavaScript
  • An overview of blockchain types

However, this chapter doesn't intend to cover:

  • The underlying cryptography
  • Cryptocurrency trading

In this chapter, the first part will introduce basic concepts. The second part will be practical, and we will discover how to interact with the blockchain, using the famous Hello World example to get you started.

 

The emergence of blockchain and cryptocurrency

Many find it hard to understand the logic and the concepts behind blockchain, and why they would need it. This is primarily because we don't have a clear idea what problems it solves, or what advantages it promises.

Therefore, I believe it is necessary to clarify from the start which problems are solved by blockchain. We will start by learning about the concept and history of cryptocurrencies.

From virtual currencies to cryptocurrency

Blockchain didn't appear out of the blue. It was the product of the evolution of fintech and virtual currencies over the last few decades.

At the end of the last century, the widespread use of the internet favored the emergence of digital currencies as an extension of electronic cash systems. Many projects were developed to create new digital currencies: E-cash, E-gold, WebMoney, and Liberty Reserve, to name just a few

Despite huge success in the 1990s, these projects had ceased to exist by the beginning of the new century, either through bankruptcy or being halted by authorities. A currency which is capable of disappearing overnight is a real financial nightmare, but this situation was inevitable due to the centralized nature of such digital currency systems.

There was also always a need for a central authority to be involved, to fight fraud and manage trust within the system.

Because of this fatal weakness, the opposite, decentralized model was presented as a solution. However, it was hard to establish trust in such environments without any central authority. This contrast made creating a reliable digital currency a disentangled Gordian Knot.

Thankfully, the progress of cryptography and the emergence of some clever solutions such as proof of work (for example, the hashcash Project—see http://hashcash.org) brought hope of breaking the deadlock.

The invention of bitcoin

In 2008, Satoshi Nakamoto rose to the challenge and unveiled a digital currency called bitcoin. This new currency effectively harnessed cryptography techniques to manage ownership and to secure the systemhence the name cryptocurrency.

Satoshi solved the aforementioned problems by introducing what he called initially a chain of blocks. In his published whitepaper (see https://bitcoin.org/bitcoin.pdf), he presented his vision for a new peer-to-peer electronic cash systembitcoinand described in detail its underlying machineryblockchain.

Bitcoin was the first reliable and distributed electronic cash system that's fully peer-to-peer, underpinned by the following basic concepts:

  • Encryption to ensure ownership and identity
  • A proof-of-work consensus mechanism for validating transactions and securing the network against double transactions
  • A transparent and shared ledger (a blockchain)
  • Pseudonymity

With the assumption that the network majority (>51%) is honest, the bitcoin system operates autonomously following the rules defined by the protocol (consensus rules) to validate a given transaction. By using a shared blockchain, each player has the ability to check the transaction's log history and the sender's solvency, then vote on whether the proceeded transaction is valid or not.

The voting depends on the overall hash-power the player puts into service to secure the network (initially, one CPU is one vote).

To use a cryptocurrency, users need to install a specific client which creates a wallet, generates cryptographic key pairs (private/public keys), and syncs the blockchain with the network. The public key is used by the client (software) to generate valid addresses, and the funds sent to a given address are controlled by the private key from which the address was calculated. In this way, we rely on secure cryptographic principles to manage ownership.

The following diagram depicts how transactions are processed in bitcoin's peer-to-peer network and added into a blockchain:

In a bitcoin network where users don't know one another, the blockchain is considered the single source of truth to which they refer to learn about previous consensus outcomes. The blockchain with the consensus protocol allows the network to manage transactions without a single point of failure.

What is blockchain?

Often confused with bitcoin, blockchain is the underlying technology used by bitcoin to operate. Concretely, it's an append-only and chronologically (timestamped) growing database, which harnesses basic cryptographic measures to protect stored transactions from being tampered with (in other words, data can't be deleted or altered).

This database, or ledger, collects and records monetary transactions validated by the network in elementary units called blocks. Once validated by the network consensus mechanism, these blocks are added to an existing sequential chain of cryptographic hash-linked blocks, to ensure the integrity of the datahence the name blockchain.

If a single bit changes in one of the linked blocks, the hash-link collapses, the chain is broken, and it will be rejected by the network.

The following diagram shows how the blockchain is replicated and processed by the members of the network to ensure that everyone has a consistent view of the transaction log. When a new block is validated, all nodes synchronize the same copy:

The diagram also shows us that blockchain implements a special data structure, consisting of linked blocks storing transactions and smart contracts. Let us take a closer look at these key elements in detail.

Blocks

If we consider blockchain to be a ledger or a book, a block can be compared to a page or a table in which we record a collection of confirmed transactions. Each block stored in the blockchain is uniquely identified by a hash, and composed of a header and a body.

The header encloses information about its creation (timestamp, Merkle root, Nonce, difficulty target, and version), and a reference to a previous block, whereas the body is a collection of the accepted transactions.

When a block is successfully validated (mined), it becomes part of the official blockchain. New bitcoins are generated in the block (a coinbase transaction) and paid to the validators (or miners).

Transactions

Transactions are the most fundamental building blocks of the blockchain system. They represent the transfer of value (cryptocurrency) within the blockchain network between two addresses.

More tangibly, they are represented by small data structures, defined by the blockchain protocol (such as bitcoin or Ethereum), which specifies their attributes (metadata, inputs, outputs, and so on), and model.

Before broadcasting the transaction, the user sending the funds signs it using their private key (managed by their wallet), and specifies the destination address. Digital signatures and public keys are used to enable network users to validate the transaction, and to check whether the sender has the right to spend the bitcoins held by a specific address.

Smart contracts

Smart contracts are one of the most exciting concepts in blockchain, representing self-executing scripts stored on the blockchain itself. The smart contract takes the blockchain concept to the next stage, enabling it to translate business logic into inviolable contract terms, which will be autonomously executed without relying on a broker, lawyer, or other intermediary.

The earliest form of a smart contract was defined in bitcoin using basic locking and unlocking scripts, but the concept evolved with the emergence of other blockchains.

Smart contracts are one of the more powerful, disruptive forces within blockchain, and are garnering more and more business attention, as described in the Gartner report Why Blockchain’s Smart Contracts Aren’t Ready for the Business World (see https://www.gartner.com/smarterwithgartner/why-blockchains-smart-contracts-arent-ready-for-the-business-world/). Gartner estimates that by 2022, smart contracts will be used by more than 25% of global organizations.

Owing to their importance, we will return later in this book to introduce you to smart contracts in leading blockchain platforms—bitcoin, Ethereum, and Hyperledger.

We have now finished describing the conceptslet's practice a little bit to understand what has been presented so far.

 

Interact with the blockchain

Blockchain as a technology has evolved rapidly, as new techniques deriving from the proliferation of blockchain projects have emerged. Hence the attempts to understand the present day blockchain machinery more closely led to the discovery of bitcoin.

Therefore, in this chapter we will adopt bitcoin as our main example. This choice is due to the fact that bitcoin is the original blockchain implementation, and almost all other projects mimic its design and mechanics.

In the following sections, we will connect to the bitcoin network and store the classic Hello World message into a blockchain. Bitcoin transactions can be used to store small amounts of data in a blockchain—allowing developers to build distributed systems on top of bitcoin, such as Colored Coins, Counterparty, Tierion, and more.

You would be surprised by the number of hidden messages stored in the bitcoin blockchain.

Getting started

In order to store our message into a blockchain, we will set up two bitcoin clients (a receiver and a sender). Then we will build a raw transaction, sending one bitcoin along with our message.

Technically speaking, one of the best-known practices for storing data in the bitcoin blockchain is to create a zero-value OP_RETURN output. As defined in bitcoin's protocol, the OP_RETURN script opcode enables us to store up to 80 bytes. You can check it out in bitcoin's code base—script/standard.h (see https://github.com/bitcoin/bitcoin/blob/0.15/src/script/standard.h):

static const unsigned int MAX_OP_RETURN_RELAY = 83;

As mentioned in the standard.h header file, the three additional bytes are for the necessary opcodes, and the remainder is for the extra message. More importantly, the OP_RETURN output can be pruned, helping to avoid bloating the blockchain in the future.

Don't worry if you feel lostwe will dive deep into bitcoin concepts such as outputs and scripting in the next chapter.

We will achieve our goal using two different methods:

  • By creating a raw transaction with an OP_RETURN output, using RPC commands and a bitcoin client
  • By writing a Node.js program to create and send the raw transaction using an online REST API

The second method will require some familiarity with the JavaScript programming language.

Running a bitcoin client for the first time

A bitcoin client is the end-user software that allows us to perform bitcoin operations (sending transactions, receiving payments, and so on). When you run one, you become part of the bitcoin network.

We have chosen two common clients: Bitcoin Core and Electrum. In our example, the sender will use Electrum and the receiver will use Bitcoin Core (the most popular bitcoin client).

For the purposes of this demonstration, I will install them on a single machine using Ubuntu 16.04.

You can install Bitcoin Core (version 15.04) using the following commands:

wget https://bitcoincore.org/bin/bitcoin-core-0.15.2/bitcoin-0.15.2-x86_64-linux-gnu.tar.gz
sudo install -m 0755 -o root -g root -t /usr/local/bin bitcoin-0.15.2/bin/*

Electrum is a lightweight wallet, which means it doesn't require you to download the entire blockchain, as we will see in the next section. Download and install the latest version of Electrum as follows:

wget https://download.electrum.org/3.2.2/Electrum-3.2.2.tar.gz
sudo apt-get install python3-setuptools python3-pyqt5 python3-pip
sudo pip3 install Electrum-3.2.2.tar.gz

Once both clients are installed, we need to synchronize them with the network.

Synchronizing the blockchain

We learned earlier that a blockchain is a transaction database duplicated by all computers on the network. We need to sync a huge amount of data (>200 GB) to enable the sending or receiving of bitcoins.

However, there are two workarounds to this:

  • Enabling pruning for a full-node client such as Bitcoin Core
  • Using a thin (SPV) client such as Electrum, which fetches blockchain information from Electrum servers instead of having a local copy

We will look at both solutions. Nonetheless, it's always advisable to use a bitcoin full-node client to benefit from the power of blockchain.

Running Bitcoin Core

Depending on your OS, you need to create the bitcoin.conf configuration file in the default data directory located under the following paths:

  • Windows: %APPDATA%\Bitcoin\
  • Mac: $HOME/Library/Application Support/Bitcoin/
  • Linux: $HOME/.bitcoin/

In Linux, create a .bitcoin directory using mkdir ~/.bitcoin, then create the bitcoin.conf file using nano ~/.bitcoin/bitcoin.conf.

Add the following lines to bitcoin.conf to define your client configuration (the comments after each # sign explain the parameters):

rpcuser=user_name            #Username for JSON-RPC connections
rpcpassword=your_password #Password Username for JSON-RPC connections
server=1 #Tells Bitcoin-Qt and bitcoind to accept JSON-RPC commands
testnet=1 #Run on the test network instead of the real bitcoin network.
prune=550 #Enables pruning mode

Once copied, press Ctrl + X, then Y, and then Enter to save the file.

Now our first client is ready to run on the testnet, which is a bitcoin network created for testing purposes that follows the same rules as a main network. It's a public network using worthless bitcoins. You can use this network to send free transactions and test your applications.

At the time of writing, a blockchain in its entirety exceeds 200 GB. Therefore, we activate pruning mode by setting the prune=<n> parameter in bitcoin.conf, with n indicating the amount of space you are willing to allocate to the blockchain in MB, with a minimum of 550 MB. Note that the data directory will exceed a few GB (2 GB in my case), because it hosts additional index and log files along with the UTXO database. The prune size only defines how many blocks will be downloaded.

It's now time to run Bitcoin Core. Open a new command line interface (CLI) window, and run the following command:

bitcoin-qt 

Bitcoin Core will start running with its standard GUI interface connected to the testnet.

For the first run, it will ask you to set the data directory, which we will set to the default. It will then automatically create a wallet for you, start syncing with the testnet, and download the blockchain:

Alternatively, you could run the bitcoin daemon in CLI mode with the following command:

bitcoind 

It's up to you to choose which mode to continue using (bitcoind or bitcoin-qt); the available RPC commands are the same. For my part, I'll continue this guide using btcoin-qt.

As Bitcoin Core starts up, it creates many subdirectories and files in the default data directory (.bitcoin), as shown in the following screenshot:

The main subdirectories are:

  • blocks: Stores actual bitcoin blocks
  • chainstate: Holds a LevelDB database for available UTXOs (short for Unspent Transaction Outputs)—in other words, a database storing how much money everyone has
  • wallet: Contains an encrypted wallet.dat file, which stores the private keys

Even if the network sync is not yet finished, you can open the blocks/ subdirectory to see the blockchain's blocks stored in raw format. Each blk00*.dat file is a collection of several raw blocks:

We will read the content of one of these files later.

More details about the content of the .bitcoin directory can be found in the official documentation at https://en.bitcoin.it/wiki/Data_directory.

While the server (bitcoind or bitcoin-qt) is running, open another Terminal. Let's generate a new address for our wallet by executing bitcoin-cli getnewaddress, as in the following screenshot:

Basically, bitcoin-cli is a tool that enables us to issue RPC commands to bitcoind or bitcoin-qt from the command line (bitcoin-qt users can also access the bitcoin RPC interface by using the Debug console, under the Help menu).

Now we have finished with Bitcoin Core, let's leave it to sync with the blockchain and move on to configuring Electrum.

Running Electrum

After you have downloaded and installed Electrum, open Electrum's testnet mode by running electrum --testnet. When you run Electrum for the first time, it will display the new wallet creation wizard. Follow these steps:

  1. Select Auto Connect in the first dialog box and click Next.
  2. Select Standard wallet and click Next.
  3. Keep selecting Next for each dialog box that appears, until you are asked to save your seed words. Copy them, then reconfirm that you've saved them correctly, as follows:
  1. In the last step, it will ask you for a password, which you can leave empty.
  2. Once finished, Electrum will generate a new wallet with plenty of new addresses. Quit the Electrum GUI, and let's continue in CLI mode. We run Electrum as a daemon process, whereby we execute the JSON/RPC commands as following:
electrum --testnet daemon 
electrum --testnet daemon load_wallet
  1. In a new Terminal window, run electrum --testnet listaddresses:

Great, now we have the necessary environment to start transacting with the public bitcoin network. That said, let's discover how a bitcoin transaction is created, exchanged and stored in the blockchain by constructing a bitcoin raw transaction, signing it, and broadcasting it to the network.

Method 1 – Building a raw transaction using Bitcoin Core

For the sake of brevity, we'll focus herein on the instructions needed to create and send raw transactions in Bitcoin Core, without lengthy explanations.

Don't worry if you don't understand all of what you read right away. In Chapter 2, Building a Bitcoin Payment System, we will explain the new concepts introduced in this section (inputs, outputs, scripts, and so on).

Funding our address

First off, we need to fund our previously created address with some bitcoins in order to make the first transaction. Thankfully, in the testnet we can use a free funding source called a bitcoin faucet, which provides worthless bitcoins for testing applications.

For this example, browse to the online faucet website at http://bitcoinfaucet.uo1.net/ or any other bitcoin's faucet websites, and get a few by providing the first address generated by Electrum and the address created by Bitcoin Core, as shown in the following screenshot:

Unspent transaction output

Now that we've sent the bitcoins from the faucet, let's check whether Bitcoin Core can see the transaction. To do that, we'll need to list the available UTXOs in both clients, using the listunspent RPC command.

With Bitcoin Core running, run the following command in your Terminal window:

bitcoin-cli listunspent

This will return the following result:

[{ }]

Initially, listunpsnet returns an empty result, because Bitcoin Core hasn't yet finished syncing the blockchain, which takes time (a few hours). For this reason, we will use Electrum instead of Bitcoin Core for the remainder of this guide, as it avoids us waiting for hours to see the received bitcoins.

However, we will go back using Bitcoin Core from time to time, as it has a powerful command line to deal with raw transactions.

Now run the same command for Electrum, as follows:

electrum --testnet listunspent

We will get a list of available entries, such as the following:

The previous command's output shows that we have a single available transaction received from the faucet, uniquely identified by its hash (prevout_hash field), with 1.1 Bitcoins.

More precisely, we have an available unspent transaction output from a previous transaction, which can be used as an input for the transaction we are willing to build, as follows:

In Bitcoin, transactions spend outputs from prior transactions, and generate new outputs that can be spent by transactions in the future. In fact, users move funds solely by spending UTXOs.

The previous diagram shows that the transaction (Transaction C) we received from the faucet consumes as inputs an existing output(output 1) created earlier by an old transaction. The same transaction creates two outputs: one for us (output 1), and the other returns back the change (output 0). The reason for this is that transaction outputs must be fully spent.

Unlike what you might have expected, in bitcoin, transactions don't update a global user balance (the account/balance model). Instead, they move bitcoins between one or more inputs and outputs (the UTXO model). The total balance is calculated by the bitcoin client as the sum of the values transferred by the received unspent transactions.

Creating the transaction

At this level, it's time to create a transaction that spends the received transaction. From the listunspent output, we have the necessary ingredients (prevout_hash and prevout_n) to construct our raw transaction. Let's see how.

First, you need to convert the hello world message into hexadecimal, using an online converter (such as https://codebeautify.org/string-hex-converter). The hexadecimal encoded form will be 68656c6c6f20776f726c64.

Then we have to use the createrawtransaction command, which creates a transaction spending the given inputs and creating new outputs. We have to pass as an argument (from the previous output) an object with the following parameters:

  • The txid of one of the available outputs
  • The vout index (prevout_n for Electrum) of the selected output
  • The hexadecimal form of the message
  • The destination address (created earlier)
  • The total number of satoshis (the smallest unit of the bitcoin currency) to send

Here we are sending one bitcoin, although you can set it to 0:

bitcoin-cli createrawtransaction "[{\"txid\":\"0791521362528725683caedf998006cf68b1cd817be1694ef0daca265d9b4252\", \"vout\": 1}]" "{\"data\":\"68656c6c6f20776f726c64\",\"2MsHsi4CHXsaNZSq5krnrpP4WShNgtuRa9U\":1.0000000}"

You'll get the following serialized long hex-encoded string, representing our raw transaction:

020000000152429b5d26cadaf04e69e17b81cdb168cf068099dfae3c6825875262135291070100000000ffffffff0200000000000000000d6a0b68656c6c6f20776f726c6400e1f5050000000017a914008051b4d96aa26269dfd36af0eb9c2b2fa894568700000000
To facilitate the usage of the previous CLI commands (and avoid manipulating long hex strings), you can assign the createrawtransaction output to a terminal variable, and use this later as an argument for the other commands. For example, we can use RAW=$ (bitcoin-cli createrawtransaction .....). The resulting hexadecimal string will be stored in the RAW variable, and accessible using $RAW.

Transaction structure

At first sight, the previous resultant hexadecimal string seems ambiguous and meaningless. The following table breaks down and examines indepth our transaction, byte by byte:

As you can see, our transaction has one input (the only unspent transaction received from the faucet), with the 0791...252 transaction id, and two outputs:

  • An OP_RETURN output with an OP_RETURN script
  • An output sending one bitcoin to the specified address

The transaction structure can be visualized by decoding back the raw transaction using the deserialize command. If you run electrum --testnet deserialize <Raw transactions>, it will output a meaningful JSON representation of our constructed transaction:

To get the same result, you can decode the raw transaction using bitcoin-cli decoderawtransaction, or by using an online decoder such as the one at https://live.blockcypher.com/btc-testnet/decodetx/.

Signing the transaction

At this point, the transaction is created, but not yet transmitted to the network. To send our transaction, we need to sign it using the bitcoin-cli signrawtransaction command. We sign the transaction using our private key (related to the receiving address) to prove to the network our ownership of the output, and therefore our authority to spend the held bitcoins.

The first step will be to extract the private key associated with the first address used to receive the bitcoins from the faucet:

electrum --testnet listaddresses | electrum --testnet  getprivatekeys -

Notice the presence of a dash at the end of the command. It will be replaced by the values returned from the pipe. As a result, you'll get a list of private keys. Copy the first one without the p2pkh prefix, as follows:

Beware, you should not share your private keys in real life. Remember that whoever has the private key can spend the received Bitcoins.

Next, we need to get scriptPubKey from the output we are willing to spend. For that, firstly, we have to retrieve the transaction from the blockchain, using electrum gettransaction --testnet "0791521362528725683caedf998006cf68b1cd817be1694ef0daca265d9b4252".

Secondly, we use the resultant raw form to get scriptPubKey, as follows:

electrum deserialize --testnet 0200000001915bf222c2e4e6ff36760168904ae102a0e968d83b3c575077d5475aa94dd9bf010000006b483045022100b129bc0fb5631aa668c48bb7a8fef0c81fec131d2f68ba430cd7cd9de0bd971b02203dabbf054790e31b4fd1b9a333881cd480c19b38a229e70f886dbb88ee4673f1012103bcf53d63d2fa14ee04d9ebb9170dfa7987298689c7e6ceb765c1d3ccd7f9ad01feffffff02d618b24a000000001976a914b9172e192d2805ea52fa975847eea0657e38fef888ac80778e06000000001976a914edcce89f510bf95606ec6a79cb28a745c039e22088ac63b31400

Unlike before, here we are loading and deserializing the received transaction from the faucet. We will get the outputs created in this transaction, as follows:

The part surrounded in red is scriptPubKey of the unspent transaction output.

A scriptPubKey can be seen in the outputs; it represents the conditions that are set for spending the outputs. The new owner can sign using the private key associated with the address receiving the output to fulfil the conditions of scriptPubKey.

The network checks whether the digital signature is valid, and if so makes it an input for the new transaction. The cryptographic partsscriptSig and scriptPubKey—are particularly complex, and will be discussed in the next chapter.

Copy scriptPubKey from the output, and pass it along the other options to the signrawtransaction command, as follows:

signrawtransaction "Raw hexstring" ( [{"txid":"id","vout":n,"scriptPubKey":"hex","redeemScript":"hex"},..] ["privatekey",..])

The second argument is a JSON array of the previous transaction outputs we are consuming, and the third argument is the private key belonging to the address that received the output. The result will be similar to the following output:

After succeeding in signing the raw transaction, it is time to send the signed transaction to the testnet.

Sending the transaction

To send the transaction into a blockchain, we submit the signed signature using the broadcast command provided by Electrum, as shown in the following screenshot:

You'll get back the hex-encoded transaction hash ID:

d3e300c2f2eedf673ab544f4c2b09063353e618ab8a0c9444e931d0145e43ded

Retrieving your message online from the blockchain

If everything goes as planned, you should have successfully stored the hello world message into bitcoin's testnet blockchain.

The following screenshot illustrates what we have done so far. We consumed an input (from a previous transaction), then created a transaction with two outputs; the first being an OP_RETURN transaction carrying our message along, the other one transferring one bitcoin (BTC):

Isn't it just fascinating? You can use a block explorer such as https://live.blockcypher.com/btc-testnet/tx/<txid> to inspect the transaction with the printed transaction hash (txid), and to retrieve your stored message.

It would be more exciting to retry the same operation using the mainnet (the original and main network for bitcoin), but then you would be dealing with real, expensive bitcoins.

Using the local blockchain

If Bitcoin Core has finished syncing the blockchain, you can locally parse the blocks to locate our transaction and read the stored message.

To open and parse the blockchain blocks, we need to install a graphical hex editor such as bless, by running sudo apt-get install bless.

Once installed, you can run it and open one of the .blk files present in the blocks directory:

As shown in the screenshot, bless will display a pane divided into three parts:

  • The left column is the offset column
  • The center column displays the blocks' hexadecimal content
  • The right column is the same line of data as in the center, with recognized text characters displayed as text and binary values represented by period characters

To locate our transaction, you can search for it by pasting the unsigned raw transaction string into the Search field. You may go through a few blk**.dat files before you find your transaction. In my case, I found it in the blk00100.dat file.

At first glance, it may not be very meaningful, but once you locate your transaction you can easily locate the message you’ve stored in the blockchain. The hello world message will be visible in the ASCII section on the right.

You can also locate the block that encompasses the transaction by searching for the previous block delimiter, called magic bytes, represented by 0b110907. Then you can, by following the structure of the block, determine the meaning of these long hexadecimal strings.

In the previous screenshot, I delimited the block with a yellow border and highlighted the blocks header field with multiple colors. I delimited our transaction and the coinbase transaction in blue and gray, respectively.

As you'll be running in prune mode, you will not be able to see my transaction, as you will have only synced newer blocks. However, you'll be able to see your transaction by following the same process.

To help you visualize the block content, the following table explains the meaning of the previously highlighted bytes:

And that's it! You can now send transactions with extra messages into the blockchain, and retrieve the data online or locally. Although this is not usually required, it may prove useful in the future.

Let's go ahead and send another raw transaction with an OP_RETURN output using a different method.

Method 2 – build a raw bitcoin transaction in JavaScript

At this point, I would guess that you want to write some code. Your wish is my command.

In this section, we will build a simple Node.Js script to perform what we have performed manually before: to send a raw transaction over the testnet. You can stop running Electrum and Bitcoin Core, as we will use an online REST API (chain.so/api) as a middle tier to interact with bitcoin's network.

By using an online API, we are losing the biggest advantage of blockchain: disintermediation. Instead of trusting our own blockchain copy, we have to trust a third party to read the data for us and send the transaction on our behalf. What would happen if the service provider provided wrong or outdated data?

Preparation

Before you start building your program, make sure you have Node.js and NPM (short for Node Package Manager) installed.

In order to create an OP_RETURN transaction, we can use one of the many available bitcoin APIs, such as:

In our example, we will use a JavaScript library called bitcoinjs-lib written for Node.js. We install the corresponding package as follows:

npm install bitcoinjs-lib --save

In the example code, we will submit requests using Node.js and the request package to access the API. Therefore, we install the following modules:

npm install request --save
npm install
request-promise --save

Similar to the first method, we will use the first address and its corresponding private key generated by Electrum to send a raw transaction carrying a hello world message programmatically.

Let's code

Start by creating a hello.js file and importing the bitcoinjs-lib and request-promise modules using the require directive as follows:

var bitcoin = require('bitcoinjs-lib');
var rp = require('request-promise');

Then we declare and define the necessary variables:

var data = Buffer.from('Hello World', 'utf8');
var testnet = bitcoin.networks.testnet;
var privateKey = 'cQx4Ucd3uXEpa3bNnS1JJ84gWn5djChfChtfHSkRaDNZQYA1FYnr';
var SourceAddress = "n3CKupfRCJ6Bnmr78mw9eyeszUSkfyHcPy";

They represent respectively:

  • The message to be embedded in the transaction
  • The network—testnet
  • The private key in WIF (short for Wallet Import Format)
  • The source address from which we spend the UTXO

Then we ask the API to provide us with the available unspent output belonging to a specific address. We read the response from the API to define the available amount and the output txid.

We also define the fee (5,000 satoshis) to pay the network (miners) for processing the transaction, as follows:

var url = "https://chain.so/api/v2/get_tx_unspent/BTCTEST/"+SourceAddress;
var DestionationAddress = '2MsHsi4CHXsaNZSq5krnrpP4WShNgtuRa9U';
var options = {
uri: url,
json: true
};

rp(options).then(function (response) {
var index = response.data.txs.length - 1;
console.log(response.data.txs[index]);
var UtxoId = response.data.txs[index].txid;
var vout = response.data.txs[index].output_no;
var amount = Number(response.data.txs[index].value*100000000);
var fee = 0.0005*100000000;
}).catch(function (err) { console.error(err);});

You can use console.log() at any point to print the received values in the console.

Now it's time to create our transaction. Inside the previous GET request, add the following lines:

const RawTransaction = new bitcoin.TransactionBuilder(testnet);
RawTransaction.addInput(UtxoId, vout);
RawTransaction.addOutput(DestionationAddress, parseInt(amount-fee));
scrypt = bitcoin.script.compile([bitcoin.opcodes.OP_RETURN,data]);
RawTransaction.addOutput(scrypt, 0);

Here we are using TransactionBuilder from bitcoinjs-lib to create our new raw transaction. Then we add the output we requested earlier from the API as input to our transaction.

We add two outputs: the first is an OP_RETURN output with 0 bitcoins, and the second is the output with 100,000,000 satoshis (one bitcoin), minus the fees.

Great! Everything is set! The only thing we have to do right now is to sign the transaction with our private key, and send it to the bitcoin blockchain:

var keyPair = bitcoin.ECPair.fromWIF(privateKeyWIF, testnet);
tx.sign(0, keyPair);

The second line—tx.sign(0, keyPair)—is because we are consuming a Pay-to-Public-Key-Hash (P2PKH) output. However, in bitcoin we have different types of transaction and addresses. The addresses beginning with 2 receive Pay-to-Script-Hash (P2SH) transactions, instead of the common P2PKH transactions received by addresses starting with m or n.

Of course, this changes the way we spend the output; therefore, we need to know the type of the output prior to signing the new transaction. For P2SH transactions, we need to use the following code instead:

const p2wpkh = bitcoin.payments.p2wpkh({ pubkey: keyPair.publicKey, network: bitcoin.networks.testnet });
const p2sh = bitcoin.payments.p2sh({ redeem: p2wpkh, network: bitcoin.networks.testnet});
RawTransaction.sign(0, keyPair, p2sh.redeem.output, null, parseInt(amount));

Lastly, we take the signed transaction in and send it to the specified network using a POST request with the API. We provide in our request a JSON object, which contains a hex representation of the signed transaction, as follows:

var Transaction=RawTransaction.build().toHex();
var Sendingoptions = { method: 'POST', url: 'https://chain.so/api/v2/send_tx/BTCTEST',
body: {tx_hex: Transaction}, json: true};

rp(Sendingoptions).then(function (response) {
var Jresponse = JSON.stringify(response);
console.log("Transaction ID:\n"+Jresponse);
}).catch(function (err) { console.error(err); });

Once you have saved the file, run it with the node hello.js command. If the raw transaction is valid and delivered successfully to the network, you will receive a message back that's similar to the following:

We get the used output details, along with a success message returning the transaction ID.

As before, we can check the transaction processing using a testnet explorer.

Congratulations, you have successfully built your first Node.js application to send bitcoins and to store data into a blockchain. Based on that, you can create advanced applications or develop your own protocol on top of the blockchain.

As a bonus, the full code is available in the following Github repository: https://github.com/bellaj/HelloWorld.

 

Types of blockchains

The blockchain initially started as the technology underpinning bitcoin. However, its early success to prove itself as a trust machine and intermediary killer put pressure on a variety of industries to both adapt to and adopt this new technology.

Consequently, many variations of Satoshi's original blockchain have been developed. In this final section, we will take a look at the major types of blockchain technologies.

Classification of blockchains

Currently, blockchain solutions can be classified into four basic models with different paradigms. The distinction between them is driven by two main criteria: the network type and the implemented access control model; in other words, who is allowed to join the peer-to-peer network (run a node) and access the blockchain records.

The following table gives an overview of the different types, with a basic description of each model:

You might be wonderingwhich one of these types of blockchain might be appropriate for your project?

The answers depend on your project's requirements. Generally, in a trustless environment (for remittance systems, proving provenance, and so on), we tend to use a public blockchain, while the other models are fit for actors who share a pre-existing trust and desire to build shared services.

Private blockchains are suitable for building faster local testing environments, or to avoid the cost of transacting, whereas the BaaS model is a suitable solution for easy deployment and high scalability.

 

Summary

This first chapter is designed to acquaint you with the key concepts behind blockchain, which you'll need to build blockchain projects.

We have seen how to interact with bitcoin's blockchain both manually and programmatically (using JavaScript). Now you can start building innovative solutions harnessing blockchain as a safe and trustworthy vault to help with checking the authenticity of documents, proving the provenance of products, asset digitization, and more.

A deep understanding of bitcoin is key to deciphering blockchain. In the next chapter, we will continue learning about bitcoin, and build an advanced payment application.

About the Authors
  • Bellaj Badr

    Bellaj Badr is an experienced security and software engineer who loves blockchain with a passion. Currently, he is the CTO at Mchain, a blockchain start-up that develops blockchain solutions for companies. Alongside his role as CTO, he acts as technical consultant, offering strategic and technical consulting to many companies worldwide. Aside from this, he is involved in many blockchain projects involving the establishment of new blockchain business-oriented protocols. Badr is a frequent speaker at developer conferences and is father to two angels.

    Browse publications by this author
  • Richard Horrocks

    Richard Horrocks is a freelance Ethereum and full-stack developer based in the UK, and holds a BA and MSc in natural sciences from the University of Cambridge. He worked for many years as a technical lead for Cisco Systems, where he worked on the operating systems of carrier-grade routing hardware, before leaving the world of IT to work as an English teacher. The advent of cryptocurrency piqued his interest sufficiently to lead him back to IT, and, since 2015, he has been working with Ethereum and other cryptocurrencies. His specialist interests are cryptoeconomics and incentive layers, with a particular focus on mechanism design and token engineering. When not in front of a computer, he enjoys yoga and falling off motorbikes.

    Browse publications by this author
  • Xun (Brian) Wu

    Xun (Brian) Wu is a senior blockchain architect and consultant. With over 20 years of hands-on experience across various technologies, including Blockchain, big data, cloud, AI, systems, and infrastructure, Brian has worked on more than 50 projects in his career. He has authored nine books, which have been published by O'Reilly, Packt, and Apress, focusing on popular fields within the Blockchain industry. The titles of his books include: Learn Ethereum (First Edition), Learn Ethereum (Second Edition), Blockchain for Teens, Hands-On Smart Contract Development with Hyperledger Fabric V2, Hyperledger Cookbook, Blockchain Quick Start Guide, Security Tokens and Stablecoins Quick Start Guide, Blockchain by Example, and Seven NoSQL Databases in a Week.

    Browse publications by this author
Latest Reviews (4 reviews total)
Seems to be a good means to start to develop BTC systems.
Recommended For You
Mastering Blockchain - Third Edition

Develop a deeper understanding of what’s under the hood of blockchain with this technical reference guide on one of the most disruptive modern technologies

By Imran Bashir
Blockchain Development for Finance Projects

A practical blockchain handbook designed to take you through implementing and re-engineering banking and financial solutions and workflows using eight step-by-step projects

By Ishan Roy
Learn Ethereum

Explore the blockchain-based decentralized platform and understand how Ethereum works with Dapps examples

By Xun (Brian) Wu and 2 more
Practical Artificial Intelligence and Blockchain

Learn how to use AI and blockchain to build decentralized intelligent applications (DIApps) that overcome real-world challenges

By Ganesh Prasad Kumble