In this article by Narayan Prusty, author of the book Building Blockchain Projects, we learn how to compile smart contracts using web3.js and deploy it using web3.js and EthereumJS.Some clients may need to compile and deploy contracts at runtime. In our proof-of-ownership DApp, we deployed the smart contract manually and hardcoded the contract address in the client-side code. But some clients may need to deploy smart contracts atruntime.For example, if a client lets schools record students' attendance in the blockchain, then it will need to deploy a smart contract every time a new school is registered so that each school has complete control over their smart contract.

In this article, we'll cover the following topics:

Calculating the nonce of a transaction

Using the transaction pool JSON-RPC API

Generating data of a transaction for contract creation and method invocation

Estimating the gas required by a transaction

Findingthe current spendablebalance of an account

Compiling smart contracts using solcjs

Developing a platform to write, compile, and deploy smart contracts

(For more resources related to this topic, see here.)

Calculating a transaction's nonce

For the accounts maintained by geth, we don't need to worry about the transaction nonce because geth can add the correct nonce to the transactions and sign them. While using accounts that aren't managed by geth, we need to calculate the nonce ourselves.

To calculate the nonce ourselves, we can use the getTransactionCount method provided by geth. The first argument should be the address whose transaction count we need and the second argument is the block till untilwe need the transaction count. We can provide the"pending" string as the block to include transactions from the block that's currently being mined.To mine a block, gethtakes the pending transactions from the transactions pool and starts mining the new block. Untilthe block is not mined, the pending transactions remain in the transaction pool and once mined, the mined transactions are removed from the transaction pools. The new incoming transactions received while a block is being mined are put in the transaction pool and are mined in the next block. So when we provide "pending" as the second argument while calling getTransactionCount, it doesn't look inside in the transaction pool; instead, it just considers the transactions in the pending block.

So if you are trying to send transactions from accounts not managed by geth, then countthe total number of transactions of the account in the blockchain and add it with the transactions pending in the transactions pool. If you try to use pending transactions from the pending block, then you will fail to get the correct nonce if transactions are sent to geth within a few seconds of the interval because it takes 12 seconds on average to include a transaction in the blockchain.

Introducingsolcjs

Solcjs is a node.js library and command-line tool thatis used to compile solidity files.Itdoesn't use the solc command-line compiler; instead, it compiles purely using JavaScript, so it's much easier to install than solc.

Solc is the actual solidity compiler. Solc is written in C++. The C++ code is compiled to JavaScript using emscripten. Every version of solc is compiled to JavaScript. Athttps://github.com/ethereum/solc-bin/tree/gh-pages/bin, you can find the JavaScript-based compilers of each solidity version. Solcjs just uses one of these JavaScript-basedcompilers to compile the solidity source code.These JavaScript-based compilers can run in both browser and NodeJS environment.

Browser solidity uses these JavaScript-based compilers to compile the solidity source code.

Installing solcjs

Solcjs is available as an npm package with the name solc. You can install thesolcjs npm package locally or globally just like any other npm package. If this package is installed globally, then solcjs, a command-line tool, will be available. So, in order to install the command-line tool, run this command:

npm install -g solc

Now go ahead and run this command to see how to compile solidity files using the command-line compiler:

solcjs –help

We won't be exploring the solcjs command-line tool; instead, we will learn about the solcjs APIs to compile solidity files.

By default, solcjs uses compiler version matching as its version. For example,if you install solcjs version 0.4.8, then it will use the 0.4.8 compiler version to compileby default. Solcjs can be configured to use some other compiler version too. At the time of writing this, the latest version of solcjs is0.4.8.

Solcjs APIs

Solcjs provides a compilermethod, which is used to compile solidity code. This method can be used in two different ways depending on whether the source code has any imports or not. If the source code doesn't have any imports, then it takes two arguments;that is,the first argument is solidity source code as a string and a Boolean indicating whether to optimize the byte code or not. If the source string contains multiple contracts, then it will compile all of them.

Here is an example to demonstrate this:

var solc = require("solc");
var input = "contract x { function g() {} }";
var output = solc.compile(input, 1); // 1 activates the optimiser 
for (var contractName in output.contracts) {
    // logging code and ABI 
    console.log(contractName + ": " + output.contracts[contractName].bytecode);
    console.log(contractName + "; " + JSON.parse(output.contracts[contractName].interface));
}

If your source code contains imports, then the first argument will be an object whose keysare filenames and valuesare the contents of the files. So whenever the compiler sees an import statement, it doesn't look for the file in the filesystem; instead, it looks for the file contents in the object by matching the filename with the keys.Here is an example to demonstrate this:

var solc = require("solc");
var input = {
"lib.sol": "library L { function f() returns (uint) { return 7; } }",
"cont.sol": "import 'lib.sol'; contract x { function g() { L.f(); } }"
};
var output = solc.compile({sources: input}, 1);
for (var contractName in output.contracts)
    console.log(contractName + ": " + output.contracts[contractName].bytecode);

If you want to read the imported file contents from the filesystem during compilation or resolve the file contents during compilation, then the compiler method supports a third argument, which is a method that takes the filename and should return the file content. Here is an example to demonstrate this:

var solc = require("solc");
var input = {
"cont.sol": "import 'lib.sol'; contract x { function g() { L.f(); } }"
};
function findImports(path) {
    if (path === "lib.sol")
        return { contents: "library L { function f() returns (uint) { return 7; } }" }
    else
        return { error: "File not found" }
}
var output = solc.compile({sources: input}, 1, findImports);
for (var contractName in output.contracts)
    console.log(contractName + ": " + output.contracts[contractName].bytecode);

Using a different compiler version

In order to compile contracts using a different version of solidity, you need to use theuseVersion method to get a reference of a different compiler. useVersion takes a string thatindicates the JavaScript filename that holds the compiler, and it looks for the file in the /node_modules/solc/bindirectory.

Solcjs also provides another method called loadRemoteVersion, which takes the compiler filename that matches the filename in the solc-bin/bin directory of the solc-binrepository (https://github.com/ethereum/solc-bin)and downloads and uses it.

Finally, solcjs also provides another method called setupMethods,which is similar to useVersion but can load the compiler from any directory.

Here is an example to demonstrate all three methods:

var solc = require("solc");

var solcV047 = solc.useVersion("v0.4.7.commit.822622cf");
var output = solcV011.compile("contract t { function g() {} }", 1);

solc.loadRemoteVersion('soljson-v0.4.5.commit.b318366e', function(err, solcV045) {
    if (err) {
        // An error was encountered, display and quit
    }

    var output = solcV045.compile("contract t { function g() {} }", 1);
});

var solcV048 = solc.setupMethods(require("/my/local/0.4.8.js"));
var output = solcV048.compile("contract t { function g() {} }", 1);

solc.loadRemoteVersion('latest', function(err, latestVersion) {
    if (err) {
        // An error was encountered, display and quit
    }
    var output = latestVersion.compile("contract t { function g() {} }", 1);
});

To run the precedingcode, you need to first download thev0.4.7.commit.822622cf.js file from thesolc-bin repository and place it in thenode_modules/solc/bin directory. And then, you need to download the compiler file of solidity version 0.4.8 and place it somewherein the filesystem and point the path in the setupMethods call to that directory.

Linking libraries

If your solidity source code references libraries, then the generated byte code will contain placeholders for the real addresses of the referenced libraries. These have to be updated via a process called linking before deploying the contract.

Solcjs provides thelinkByteCode method to link library addresses to the generated byte code.

Here is an example to demonstrate this:

var solc = require("solc");

var input = {
"lib.sol": "library L { function f() returns (uint) { return 7; } }",
"cont.sol": "import 'lib.sol'; contract x { function g() { L.f(); } }"
};

var output = solc.compile({sources: input}, 1);

var finalByteCode = solc.linkBytecode(output.contracts["x"].bytecode, { 'L': '0x123456...' });

Updating the ABI

The ABI of a contract provides various kinds of information about the contract other than implementation.ABI generated by two different versions of compilers may not match as higher versions support more solidity features than lower versions; therefore, they will include extra things in the ABI.For example, thefallback function was introduced in 0.4.0 version of Solidity so the ABI generated using compilers whose version is less than 0.4.0 will have no information aboutfallback functions, and these smart contracts behave like they havea fallback function with an empty body and apayable modifier. So, the API should be updated so that applications thatdepend onthe ABI of newer solidity version can have better information about the contract.

Solcjs provides an API to update ABI. Here is an example code to demonstrate this:

var abi = require("solc/abi");

var inputABI = [{"constant":false,"inputs":[],"name":"hello","outputs":[{"name":"","type":"string"}],"payable":false,"type":"function"}];
var outputABI = abi.update("0.3.6", inputABI)

Here, 0.3.6 indicates that the ABI was generated using 0.3.6 version of compiler. As we are using solcjs version 0.4.8, the ABI will be updated to match the ABI generated by 0.4.8 compiler version not above it.

The output of the precedingcode will be as follows:

[{"constant":false,"inputs":[],"name":"hello","outputs":[{"name":"","type":"string"}],"payable":true,"type":"function"},{"type":"fallback","payable":true}]

Building a contract deployment platform

Now that we have learned how to use solcjs to compile solidity source code,it's time to build a platform that lets us write, compile, and deploy contracts.Our platform will let users provide their account address and private key, using which our platform will deploy contracts.

Before you start building the application, make sure that you are running the geth development instance, which is mining, has rpc enabled, and exposeseth, web3, and txpool APIs over the HTTP-RPC server. You can do all these by running this:

geth --dev --rpc --rpccorsdomain "*" --rpcaddr "0.0.0.0" --rpcport "8545" --mine --rpcapi "eth,txpool,web3"

Building the backend

Let's first build the backend of the app. First of all, run npm install inside the Initial directory to install the required dependencies for our backend.

Here is the backend code to run an express service and serve the index.html file and static files:

var express = require("express");  
var app = express();  

app.use(express.static("public"));

app.get("/", function(req, res){
	res.sendFile(__dirname + "/public/html/index.html");
})

app.listen(8080);

The precedingcode is self-explainable.Now let's proceed further.Our app will have two buttons,that is, compile and deploy. When the user will click on the compile button, the contract will be compiled and when the deploy button is clicked on, the contract will be deployed.

We will be compiling and deploying contracts in the backend. Although this can be done in the frontend, we will do it in the backend because solcjs is available only for NodeJS (although the JavaScript-based compilers it uses work on the frontend).

To learn how to compile on the frontend, go through the source code of solcjs, which will give you an idea about the APIs exposed by the JavaScript-based compiler.

When the user clickson the compile button, the frontend will make a GET request to the /compile path by passing the contract source code. Here is the code for the route:

var solc = require("solc");

app.get("/compile", function(req, res){
	var output = solc.compile(req.query.code, 1);
	res.send(output);
})

At first, we import the solcjslibrary here. Then, we define the /compile route and inside the route callback, we simply compile the source code sent by the client with the optimizer enabled.And then we just send the solc.compile method's return value to the frontend and let the client checkwhether the compilation was successful or not.

When the user clicks on the deploy button, the frontend will make a GET request to the /deploy path by passing the contract source code and constructor arguments from the address and private key.When the user clicks on this button, the contract will be deployed and the transaction hash will be returned to the user.

Here is the code for this:

var Web3 = require("web3");
var BigNumber = require("bignumber.js");
var ethereumjsUtil = require("ethereumjs-util");
var ethereumjsTx = require("ethereumjs-tx");

var web3 = new Web3(new Web3.providers.HttpProvider("http://localhost:8545"));

function etherSpentInPendingTransactions(address, callback)
{
	web3.currentProvider.sendAsync({
		method: "txpool_content",
		params: [],
		jsonrpc: "2.0",
		id: new Date().getTime()
	}, function (error, result) {
		if(result.result.pending)
		{
			if(result.result.pending[address])
			{
				var txns = result.result.pending[address];
				var cost = new BigNumber(0);
				
				for(var txn in txns)
				{
					cost = cost.add((new BigNumber(parseInt(txns[txn].value))).add((new BigNumber(parseInt(txns[txn].gas))).mul(new BigNumber(parseInt(txns[txn].gasPrice)))));
				}

				callback(null, web3.fromWei(cost, "ether"));
			}
			else
			{
				callback(null, "0");
			}
		}
		else
		{
			callback(null, "0");
		}
	})
}

function getNonce(address, callback)
{
	web3.eth.getTransactionCount(address, function(error, result){
		var txnsCount = result;

		web3.currentProvider.sendAsync({
			method: "txpool_content",
			params: [],
			jsonrpc: "2.0",
			id: new Date().getTime()
		}, function (error, result) {
			if(result.result.pending)
			{
				if(result.result.pending[address])
				{
					txnsCount = txnsCount + Object.keys(result.result.pending[address]).length;
					callback(null, txnsCount);
				}
				else
				{
					callback(null, txnsCount);
				}
			}
			else
			{
				callback(null, txnsCount);
			}
		})
	})
}

app.get("/deploy", function(req, res){
	var code = req.query.code;
	var arguments = JSON.parse(req.query.arguments);
	var address = req.query.address;

	var output = solc.compile(code, 1);

	var contracts = output.contracts;

	for(var contractName in contracts)
	{
		var abi = JSON.parse(contracts[contractName].interface);
		var byteCode = contracts[contractName].bytecode;

		var contract = web3.eth.contract(abi);

		var data = contract.new.getData.call(null, ...arguments, {
			data: byteCode
		});

		var gasRequired = web3.eth.estimateGas({
		    data: "0x" + data
		});

		web3.eth.getBalance(address, function(error, balance){
			var etherAvailable = web3.fromWei(balance, "ether");
			etherSpentInPendingTransactions(address, function(error, balance){
				etherAvailable = etherAvailable.sub(balance)
				if(etherAvailable.gte(web3.fromWei(new BigNumber(web3.eth.gasPrice).mul(gasRequired), "ether")))
				{
					getNonce(address, function(error, nonce){
						var rawTx = {
					        gasPrice: web3.toHex(web3.eth.gasPrice),
					        gasLimit: web3.toHex(gasRequired),
					        from: address,
					        nonce: web3.toHex(nonce),
					        data: "0x" + data
					    };

						var privateKey = ethereumjsUtil.toBuffer(req.query.key, 'hex');
						var tx = new ethereumjsTx(rawTx);
						tx.sign(privateKey);

						web3.eth.sendRawTransaction("0x" + tx.serialize().toString('hex'), function(err, hash) {
							res.send({result: {
								hash: hash,
							}});
						});
					})
				}
				else
				{
					res.send({error: "Insufficient Balance"});
				}
			})
		})

	break;
	}
})

This is how the preceding code works:

At first, the Web imported the web3.js, BigNumber.js, ethereumjs-util, and ethereumjs-tx libraries. Then, we created an instance of Web3.

Then, we defined a function named etherInSpentPendingTransactions,which calculatesthe total ether that's being spent in the pending transactions of an address. As web3.js doesn't provide JavaScript APIs related to the transaction pool, we make a raw JSON-RPC call using web3.currentProvider.sendAsync.sendAsync is used to make raw JSON-RPC calls asynchronously. If you want to make this call synchronously, then use thesend method instead of sendAsync. While calculating the total ether in the pending transactions of an address, we look for pending transactions in the transactions pool instead of the pending block due to the issue we discussed earlier. While calculating the total ether, we add the value and gas of each transaction as gas also deducted ether balance.

Next, we defined a function called getNonce, which retrieves the nonce of an address using the technique we discussed earlier. It simply adds the total number of mined transactions to the total number of pending transactions.

Finally, we declared the /deploy endpoint.At first, we compile the contract. Then, we deployonly the first contract. Our platform is designed to deploy the firstcontractif multiple contracts are found in the provided source code. You can later enhance the app to deploy all the compiled contracts instead of just the first one.Then, we created a contract object using web3.eth.contract.

As we aren't using hooked-web3-provider or any hack to intercept sendTransactions and convert them into thesendRawTransaction call,in order to deploy the contract, we now need to generate the data part of the transaction, which will have the contract byte code and constructor arguments combined and encoded as a hexadecimal string. The contract object actually lets us generate the data of the transaction. This can be done by calling thegetData method with function arguments. If you want to get data todeploy the contract, then call contract.new.getData, and if you want to call a function of the contract, then call contract.functionName.getData. In both the cases, provide the arguments to the getData method. So, in order to generate the data of a transaction, you just need the contract's ABI. To learn how the function name and arguments are combined and encoded to generate data, you can check outhttps://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI#examples,but this won't be required if you have the ABI of the contract or know how to create ABI manually.

Then, we useweb3.eth.estimateGas to calculate the amount of gas that would be required to deploy the contract.

Later, we check whether the address has enough ether to pay for the gas required to deploy the contract. We find this out by retrieving the balance of the address and subtracting it with the balance spent in the pending transactions and then checking whetherthe remaining balance is greater thanor equal to the amount of ether required for the gas.

And finally, we get the nonce, signing and broadcasting the transactions. We simply return the transaction hash to the frontend.

Building the frontend

Now let's build the frontend of our application. Our frontend will contain an editor, using which the user writes code. And when the user clicks on the compile button, we will dynamically display input boxes where each input box will represent a constructor argument. When the deploy button is clicked on, the constructor argument value are taken from these input boxes. The user will need to enter the JSON string in these input boxes.

We will be using the codemirror library to integrate the editor in our frontend. To learn more about how to use codemirror, refer to http://codemirror.net/.

Here is the frontend HTML code of our app. Place this code in the index.html file:

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta http-equiv="x-ua-compatible" content="ie=edge">
<link rel="stylesheet" href="/css/bootstrap.min.css">
<link rel="stylesheet" href="/css/codemirror.css">
<style type="text/css">
            .CodeMirror
            {
                height: auto;
            }
</style>
</head>
<body>
<div class="container">
<div class="row">
<div class="col-md-6">
<br>
<textarea id="editor"></textarea>
<br>
<span id="errors"></span>
<button type="button" id="compile" class="btn btn-primary">Compile</button>
</div>
<div class="col-md-6">
<br>
<form>
<div class="form-group">
<label for="address">Address</label>
<input type="text" class="form-control" id="address" placeholder="Prefixed with 0x">
</div>
<div class="form-group">
<label for="key">Private Key</label>
<input type="text" class="form-control" id="key" placeholder="Prefixed with 0x">
</div>
<hr>
<div id="arguments"></div>
<hr>
<button type="button" id="deploy" class="btn btn-primary">Deploy</button>
</form>
</div>
</div>
</div>
<script src="/js/codemirror.js"></script>
<script src="/js/main.js"></script>
</body>
</html>

Here, you can see that we have a textarea. The textareatag will hold whatever user will enter in the codemirror editor. Everything else in the precedingcode is self-explanatory.

Here is the complete frontend JavaScript code. Place this code in the main.js file:

var editor = CodeMirror.fromTextArea(document.getElementById("editor"), {
    lineNumbers: true,
});

var argumentsCount = 0;

document.getElementById("compile").addEventListener("click", function(){
	editor.save();
	var xhttp = new XMLHttpRequest();
	
	xhttp.onreadystatechange = function() {
	    if (this.readyState == 4 && this.status == 200) {
		if(JSON.parse(xhttp.responseText).errors != undefined)
		{
			document.getElementById("errors").innerHTML = JSON.parse(xhttp.responseText).errors + "<br><br>";
		}
		else
		{
			document.getElementById("errors").innerHTML = "";
		}

		var contracts = JSON.parse(xhttp.responseText).contracts;

		for(var contractName in contracts)
		{
			var abi = JSON.parse(contracts[contractName].interface);

			document.getElementById("arguments").innerHTML = "";

			for(var count1 = 0; count1 < abi.length; count1++)
			{
				if(abi[count1].type == "constructor")
				{
					argumentsCount = abi[count1].inputs.length;

					document.getElementById("arguments").innerHTML = '<label>Arguments</label>';

					for(var count2 = 0; count2 < abi[count1].inputs.length; count2++)
					{
						var inputElement = document.createElement("input");
							inputElement.setAttribute("type", "text");
							inputElement.setAttribute("class", "form-control");
							inputElement.setAttribute("placeholder", abi[count1].inputs[count2].type);
							inputElement.setAttribute("id", "arguments-" + (count2 + 1));

							var br = document.createElement("br");

							document.getElementById("arguments").appendChild(br);
							document.getElementById("arguments").appendChild(inputElement);
					}

					break;
				}
			}

			break;
		}
	    }
	};

	xhttp.open("GET", "/compile?code=" + encodeURIComponent(document.getElementById("editor").value), true);
	xhttp.send();	
})

document.getElementById("deploy").addEventListener("click", function(){
	editor.save();

	var arguments = [];

	for(var count = 1; count <= argumentsCount; count++)
	{
		arguments[count - 1] = JSON.parse(document.getElementById("arguments-" + count).value); 
	}

	var xhttp = new XMLHttpRequest();
	
	xhttp.onreadystatechange = function() {
	    if (this.readyState == 4 && this.status == 200) 
	    {
		var res = JSON.parse(xhttp.responseText);

		if(res.error)
		{
			alert("Error: " + res.error)
		}
		else
		{
			alert("Txn Hash: " + res.result.hash);
		} 
	    }
	    else if(this.readyState == 4)
	    {
		alert("An error occured.");
	    }
	};

	xhttp.open("GET", "/deploy?code=" + encodeURIComponent(document.getElementById("editor").value) + "&arguments=" + encodeURIComponent(JSON.stringify(arguments)) + "&address=" + document.getElementById("address").value + "&key=" + document.getElementById("key").value, true);
	xhttp.send();	
})

Here is how the precedingcode works:

At first, we add the code editor to the webpage. The code editor will be displayed in place of textarea and textarea will be hidden.

Then we have the compile button's click event handler. Inside it, we save the editor, which copies the content of the editor to textarea.When the compile button is clicked on, we make a request to the /compile path, and once we get the result, we parse it and display the input boxes so that the user can enter the constructor arguments. Here, we only read the constructor arguments for the first contract. But you can enhance the UI to display input boxes for constructors of all the contracts if there are more than one.

And finally, we have the deploy button's click event handler.Here,we read the constructor arguments' value, parsing and putting them in an array. And then we adda request to the /deploy endpoint by passing the address, key, code, and argument value.If there is an error, then we display that in a popup; otherwise, we display the transaction hash in the popup.

Testing

To test the app, run the app.jsnode inside the Initial directory and visit localhost:8080. You will see what is shown in the following screenshot:

building-smart-contract-deployment-platform-img-0

Now enter some solidity contract code and press the compile button. Then, you will be able to see new input boxes appearing on the right-hand side. For example,take a look at the following screenshot:

Now enter a valid address and its associated private key. And then enter values for the constructor arguments and click on deploy. If everything goes right, then you will see an alert box with transaction hash. For example, take a look at the following screenshot:

building-smart-contract-deployment-platform-img-2

Summary

In this article, we learned how to use the transaction pool API, how to calculate proper nonce, calculate spendable balance, generate data of a transaction, compile contracts, and so on. We then built a complete contract compilation and deployment platform. Now you can go ahead and enhance the application we have built to deploy all the contracts found in the editor, handle imports, add libraries, and so on.