Deploying Node.js

Running code procedurally, or in order, is a reasonable idea. We tend to do that when we execute tasks and, for a long time, programming languages were naturally procedural. Clearly, at some point, the instructions you send to a processor must be executed in a predictable order. If I want to multiply 8 by 6, divide that result by 144 divided by 12, and then add the total result to 10, the order of those operations must proceed sequentially:

( (8x6) / (144/12) ) + 10

The order of operations must not be as follows:

(8x6) / ( (144/12) + 10 )

This is logical and easy to understand. Early computers typically had one processor, and processing one instruction blocked the processing of subsequent instructions. But things did not stay that way, and we have moved far beyond single-core computers.

If you think about the previous example, it should be obvious that calculating 144/12 and 8x6 can be done independently—one need not wait for the other. A problem can be divided into smaller problems and distributed across a pool of available people or workers to work on in parallel, and the results can be combined into a correctly ordered final calculation.

Multiple processes, each solving one part of a single mathematical problem simultaneously, are an example of parallelism.

Rob Pike, co-inventor of Google's Go programming language, defines concurrency in this way:

Concurrency is not parallelism. A system demonstrating concurrency allows developers to compose applications as if multiple independent processes are simultaneously executing many possibly related things. Successful high-concurrency application development frameworks provide an easy-to-reason-about vocabulary to describe and build such a system.

Node's design suggests that achieving its primary goal—to provide an easy way to build scalable network programs—includes simplifying how the execution order of coexisting processes is structured and composed. Node helps a developer reasoning about a program, within which many things are happening at once (such as serving many concurrent clients), to better organize his or her code.

Let's take a look at the differences between parallelism and concurrency, threads and processes, and the special way that Node absorbs the best parts of each into its own unique design.

The following diagram describes how a traditional microprocessor might execute the simple program discussed previously:

The program is broken up into individual instructions that are executed in order. This works but does require that instructions be processed in a serial fashion, and, while any one instruction is being processed, subsequent instructions must wait. This is a blocking process—executing any one segment of this chain blocks the execution of subsequent segments. There is a single thread of execution in play.

However, there is some good news. The processor has (literally) total control of the board, and there is no danger of another processor nulling memory or overriding any other state that this primary processor might manipulate. Speed is sacrificed for stability and safety.

We do like speed; however, the model discussed earlier rapidly became obsolete as chip designers and systems programmers worked to introduce parallel computing. Rather than having one blocking thread, the goal was to have multiple cooperating threads.

This improvement definitely increased the speed of calculation but introduced some problems, as described in the following schematic:

This diagram illustrates cooperating threads executing in parallel within a single process, which reduces the time necessary to perform the given calculation. Distinct threads are employed to break apart, solve, and compose a solution. As many subtasks can be completed independently, the overall completion time can be reduced dramatically.

Threads provide parallelism within a single process. A single thread represents a single sequence of (serially executed) instructions. A process can contain any number of threads.

Difficulties arise out of the complexity of thread synchronization. It is very difficult to model highly concurrent scenarios using threads, especially models in which the state is shared. It is difficult to anticipate all the ways in which an action taken in one thread will affect all the others if it is never clear when an asynchronously executing thread will complete:

We want the power of parallelization provided by threads but could do without the mind-bending world of semaphores and mutexes. In the Unix world, there is a concept that is sometimes referred to as the Rule of Simplicity: Developers should design for simplicity by looking for ways to break up program systems into small, straightforward cooperating pieces. This rule aims to discourage developers' affection for writing 'intricate and beautiful complexities' that are, in reality, bug-prone programs.

Parallelism within a single process is a complicated illusion that is achieved deep within mind-bendingly complex chipsets and other hardware. The question is really about appearances—about how the activity of the system appears to, and can be programmed by, a developer. Threads offer hyper-efficient parallelism, but make concurrency difficult to reason about.

Rather than have the developer struggle with this complexity, Node itself manages I/O threads, simplifying this complexity by demanding only that control flow be managed between events. There is a need to micromanage I/O threading; one simply designs an application to establish data availability points (callbacks) and the instructions to be executed once the said data is available. A single stream of instructions that explicitly takes and relinquishes control in a clear, collision-free, and predictable way aids development:

A single thread describing asynchronous control flow efficiently managed by an event loop brings stability, maintainability, readability, and resilience to Node programs. The big news is that Node continues to deliver the speed and power of multithreading to its developers—the brilliance of Node's design makes such power transparent, reflecting one part of Node's stated aim of bringing the most power to the most people with the least difficulty.

Many JavaScript extensions in Node emit events. These are instances of events.EventEmitter. Any object can extend EventEmitter, which gives the developer an elegant toolkit to build tight, asynchronous interfaces to their object methods.

Work through this example demonstrating how to set an EventEmitter object as the prototype of a function constructor. As each constructed instance now has the EventEmitter object exposed to its prototype chain, this provides a natural reference to the event's Application Programming Interface (API). The counter instance methods can, therefore, emit events, and these can be listened for. Here, we emit the latest count whenever the counter.increment method is called and bind a callback to the "incremented" event, which simply prints the current counter value to the command line:

var EventEmitter = require('events').EventEmitter;
var util = require('util');

var Counter = function(init) {
  this.increment = function() {
    init++;
    this.emit('incremented', init);
  }
}

util.inherits(Counter, EventEmitter);

var counter = new Counter(10);

var callback = function(count) {
  console.log(count);
}
counter.addListener('incremented', callback);

counter.increment(); // 11
counter.increment(); // 12

To remove the event listeners bound to counter, use counter.removeListener('incremented', callback).

EventEmitter, as an extensible object, adds to the expressiveness of JavaScript. For example, it allows I/O data streams to be handled in an event-oriented manner in keeping with Node's principle of asynchronous, nonblocking programming:

var stream = require('stream');
var Readable = stream.Readable;
var util = require('util');

var Reader = function() {
  Readable.call(this);
  this.counter = 0;
}

util.inherits(Reader, Readable);

Reader.prototype._read = function() {
  if(++this.counter > 10) {
    return this.push(null);
  }
  this.push(this.counter.toString());
};

// When a #data event occurs, display the chunk.
//
var reader = new Reader();
reader.setEncoding('utf8');
reader.on('data', function(chunk) {
  console.log(chunk);
});
reader.on('end', function() {
  console.log('--finished--');
});

In this program, we have a Readable stream pushing out a set of numbers—with listeners on that stream's data event catching numbers as they are emitted and logging them—and finishing with a message when the stream has ended. It is plain that the listener is called once per number, which means that running this set did not block the event loop. Because Node's event loop need only commit resources to handling callbacks, many other instructions can be processed in the downtime of each event.

The code seen in non-networked software is often synchronous or blocking. I/O operations in the following pseudo-code are also blocking:

variable = produceAValue()
print variable
// some value is output when #produceAValue is finished.

The following iterator will read one file at a time, dump its contents, and then read the next until it is done:

fileNames = ['a','b','c']
while(filename = fileNames.shift()) {
  fileContents = File.read(filename)
  print fileContents
}
//	> a
//	> b
//	> c

This is a fine model for many cases. However, what if these files are very large? If each takes 1 second to fetch, all will take 3 seconds to fetch. The retrieval on one file is always waiting on another retrieval to finish, which is inefficient and slow. Using Node, we can initiate file reads on all files simultaneously:

var fs = require('fs');
var fileNames = ['a','b','c'];
fileNames.forEach(function(filename) {
  fs.readFile(filename, {encoding:'utf8'}, function(err, content) {
    console.log(content);
  });
});
//	> b
//	> a
//	> c

The Node version will read all three files at once, each call to fs.readFile returning its result at some unknowable point in the future. This is why we can't always expect the files to be returned in the order they were arrayed. We can expect that all three will be returned in roughly the time it took for one to be retrieved—something less than 3 seconds. We have traded a predictable execution order for speed, and, as with threads, achieving synchronization in concurrent environments requires extra work. How do we manage and describe unpredictable data events so that our code is both easy to understand and efficient?

The key design choice made by Node's designers was the implementation of an event loop as a concurrency manager. The following description of event-driven programming (taken from http://www.princeton.edu/~achaney/tmve/wiki100k/docs/Event-driven_programming.html) clearly not only describes the event-driven paradigm, but also introduces us to how events are handled in Node and how JavaScript is an ideal language for such a paradigm:

As we've seen in the preceding quote, single-threaded execution environments block and can, therefore, run slowly. V8 provides a single thread of execution for JavaScript programs.

How can this single thread be made more efficient?

Node makes a single thread more efficient by delegating many blocking operations to OS subsystems to process, bothering the main V8 thread only when there is data available for use. The main thread (your executing Node program) expresses interest in some data (such as via fs.readFile) by passing a callback and is notified when that data is available. Until that data arrives, no further burden is placed on V8's main JavaScript thread. How? Node delegates I/O work to libuv, as quoted at http://nikhilm.github.io/uvbook/basics.html#event-loops:

The user in the preceding quote is the Node process executing a JavaScript program. Callbacks are JavaScript functions, and managing callback invocation for the user is accomplished by Node's event loop. Node manages a queue of I/O requests populated by libuv, which is responsible for polling the OS for I/O data events and handing off the results to JavaScript callbacks.

Consider the following code:

var fs = require('fs');
fs.readFile('foo.js', {encoding:'utf8'}, function(err, fileContents) {
  console.log('Then the contents are available', fileContents);
});
console.log('This happens first');

This program will result in the following output:

> This happens first
> Then the contents are available, [file contents shown]

Here's what Node does when executing this program:

Here, we see the key ideas that Node implements to achieve fast, manageable, and scalable I/O. If, for example, there were 10 read calls made for 'foo.js' in the preceding program, the execution time would, nevertheless, remain roughly the same. Each call would have been made in parallel in its own thread within the libuv thread pool. Even though we wrote our code "in JavaScript", we are actually deploying a very efficient multithreaded execution engine while avoiding the difficulties of thread management.

Let's close with more details on how exactly libuv results are returned into the main thread's event loop.

When data becomes available on a socket or other stream interface, we cannot simply execute our callback immediately. JavaScript is single threaded, so results must be synchronized. We can't suddenly change the state in the middle of an event loop tick—this would create some of the classic multithreaded application problems of race conditions, memory access conflicts, and so on.

Upon entering an event loop, Node (in effect) makes a copy of the current instruction queue (also known as stack), empties the original queue, and executes its copy. The processing of this instruction queue is referred to as a tick. If libuv, asynchronously, receives results while the chain of instructions copied at the start of this tick are being processed on the single main thread (V8), these results (wrapped as callbacks) are queued. Once the current queue is emptied and its last instruction has completed, the queue is again checked for instructions to execute on the next tick. This pattern of checking and executing the queue will repeat (loop) until the queue is emptied, and no further data events are expected, at which point the Node process exits.

The following are the sorts of I/O events fed into the queue:

There are two important things to remember:

To learn more about how Node is bound to libuv and other core libraries, parse through the fs module code at https://github.com/joyent/node/blob/master/lib/fs.js. Compare the fs.read and the fs.readSync methods to observe the difference between how synchronous and asynchronous actions are implemented—note the wrapper callback that is passed to the native binding.read method in fs.read.

To take an even deeper dive into the very heart of Node's design, including the queue implementation, read through the Node source at https://github.com/joyent/node/tree/master/src. Follow MakeCallback within fs_event_wrap.cc and node.cc. Investigate the req_wrap class, a wrapper for the V8 engine, deployed in node_file.cc and elsewhere and defined in req_wrap.h.

In his book, The Art of Unix Programming, Eric Raymond proposed the Rule of Modularity:

This idea of building complex systems out of "small pieces, loosely joined" is seen in management theory, theories of government, manufacturing, and many other contexts. In terms of software development, it advises developers to contribute only the simplest, most useful component necessary within a larger system. Large systems are hard to reason about, especially if the boundaries of their components are fuzzy.

One of the primary difficulties when constructing scalable JavaScript programs is the lack of a standard interface to assemble a coherent program out of many smaller ones. For example, a typical web application might load dependencies using a sequence of <script> tags in the <head> section of a HyperText Markup Language (HTML) document:

<head>
  <script src="fileA.js"></script>
  <script src="fileB.js"></script>
</head>

There are many problems with this sort of system:

Ambivalently inserting unpredictable code fragments into an application frustrates attempts to predictably shape functionality. What is needed is a standard way to load and share discreet program modules.

Accordingly, Node introduced the concept of the package, following the CommonJS specification. A package is a collection of program files bundled with a manifest file describing the collection. Dependencies, authorship, purpose, structure, and other important metadata is exposed in a standard way. This encourages the construction of large systems from many small, interdependent systems. Perhaps, even more importantly, it encourages sharing:

In many ways, the success of Node is due to the growth in the number and quality of packages available to the developer community that are distributed via Node's package management system, npm. This system has done much to help make JavaScript a viable, professional option for systems programming.

In his book, The C++ Programming Language, Third Edition, Bjarne Stoustrup states:

It shouldn't surprise anyone that a design team focused on providing efficient and easy I/O has delivered such a facility through Node. Through a symmetrical and simple interface, which handles data buffers and stream events so that the implementer does not have to, Node's Stream module is the preferred way to manage asynchronous data streams for both internal modules and, hopefully, for the modules that developers will create.

A stream in Node is simply a sequence of bytes or, if you like, a sequence of characters. At any time, a stream contains a buffer of bytes, and this buffer has a length of zero or more.

Because each character in a stream is well defined, and because every type of digital data can be expressed in bytes, any part of a stream can be redirected, or piped, to any other stream, different chunks of the stream can be sent to different handlers. In this way, stream input and output interfaces are both flexible and predictable and can be easily coupled.

In addition to events, Node is distinctive for its comprehensive use of streams. Continuing the idea of composing applications out of many small processes emitting events or reacting to events, several Node I/O modules and features are implemented as streams. Network sockets, file readers and writers, stdin and stdout, Zlib, and so on, are all data producers and/or consumers that are easily connected through the abstract Stream interface. Those familiar with Unix pipes will see some similarities.

Five distinct base classes are exposed via the abstract Stream interface: Readable, Writable, Duplex, Transform, and PassThrough. Each base class inherits from EventEmitter, which we know to be an interface to which event listeners and emitters can be bound. Streams in Node are evented streams, and sending data between processes is commonly done using streams. Because streams can be easily chained and otherwise combined, they are fundamental tools for the Node developer.

It is recommended that you develop a clear understanding of what streams are and how they are implemented in Node before going further as we will use streams extensively throughout this book.

Because your clients and servers will speak the same language, each can pass code to be natively executed on the other. If you are building a web application, this opens up very interesting (and unique) opportunities.

For example, consider an application that allows one client to make changes to another's environment. This tool allows a software developer to make changes to the JavaScript powering a website and allows their clients to see those changes in real time in their browsers. What this application must do is transform live code in many browsers so that it reflects changes. One way to do this would be to capture a change set into a transform function, pass that function across to all connected clients, and have that function executed in their local environment, updating it to reflect the canonical view. One application evolves, it emits a genetic update in the code of JavaScript, and the rest of its species similarly evolves. We will use one such technology in Chapter 7, Deploying and Maintaining.

Since Node shares the same JavaScript code base, a Node server, on its own initiative, can take this action. The network itself can broadcast code for its clients to execute. Similarly, clients can send code to the server for execution. It is easy to see how this allows hot code pushes, where a Node process sends a unique packet of raw JavaScript to specific clients for execution.

When Remote Procedure Calls (RPC) no longer require a broker layer to translate between communicating contexts, code can exist anywhere in the network for as long or as brief a period as necessary and can execute in multiple contexts, which are chosen for reasons of load balancing, data awareness, computational power, geographic precision, and so on.

JavaScript is the language common to Node and the browser. However, Node significantly extends the JavaScript language, adding many commands and other constructs that are not available to the client-side developer. For example, there is no equivalent of the core Node Stream module in JavaScript.

Additionally, the npm repository is rapidly growing, and, at the time of writing, contains more than 80,000 Node packages. Many of these packages are equally useful on the client as well as within Node. The spread of JavaScript to the server has, in effect, created two cooperating threads producing enterprise-grade JavaScript libraries and modules.

Browserify was developed to make it easy to share npm modules and core Node modules seamlessly with the client. Once a package has been browserified, it is easily imported into a browser environment using the standard <script> tag. Installing Browserify is simple:

npm install -g browserify

Let's build an example. Create a file, math.js, written as you would write an npm module:

module.exports = function() {
  this.add = function(a, b) {
    return a + b;
  }
  this.subtract = function(a, b) {
    return a - b;
  }
};

Next, create a program file, add.js, that uses this module:

var Math = require('./math.js');
var math = new Math;

console.log(math.add(1,3)); // 4

Executing this program using Node on the command line (> node add.js ) will result in 4 being printed to your terminal. What if we wanted to use our math module in the browser? Client-side JavaScript doesn't have a require statement, so we browserify it:

browserify math.js -o bundle.js

Browserify walks through your code, finding require statements and automatically bundling those dependencies (and the dependencies of those dependencies) into one file that you load into your client application:

<script src="bundle.js"></script>

As an added bonus, this bundle automatically introduces some useful Node globals to your browser environment: __filename, __dirname, process, Buffer, and global. This means you have, for example, process.nextTick available in the browser.