Node.js is still relatively new compared to platforms such as .NET and Java, but has become very popular in a short time, and has even started influencing these platforms. This is thanks to its distinctive programming model, extensive ecosystem, and powerful tooling.
These factors make Node.js a compelling alternative to other platforms. They can also make it intimidating. Its distinctive programming model may seem quite alien compared to other platforms. The sheer range of available libraries and tools can be bewildering.
This book will guide you through Node.js so you can start using it in your applications. It will help you to understand Node.js, navigate its ecosystem, and leverage your existing development skills in this new environment.
In this chapter, we will cover the following topics:
Introducing the Node.js platform
Seeing how its execution model works
Exploring the Node.js ecosystem
Looking at JavaScript as a language choice
Considering the range of use cases for Node.js
Node.js consists of a JavaScript engine together with low-level APIs for core server-side functionality. The execution engine is the same V8 engine developed for the Chrome web browser. Node.js takes this engine and embeds it in a standalone application that can run JavaScript outside the browser.
In Node.js, the standard APIs found in browsers to support client-side web development, such as the Document Object Model (DOM) and XMLHttpRequest, are not present. Instead, there are APIs to support general-purpose application development. These core APIs cover low-level functionality such as the following:
Networking and security
Accessing the file system
Defining and requiring modules
Raising and consuming events
Handling binary data streams
Compression
UTF-8 support
Retrieving basic information about the OS
Managing child processes
Some of these APIs may already be familiar from developing client-side JavaScript. For example, the Timers API exposes the familiar setTimeout and setInterval functions.
Node.js also provides several tools to help with the development process. These include console logging, debugging, a Read-Eval-Print Loop (REPL) (or interactive console), and basic assertions for testing.
The execution model of Node.js follows that of JavaScript in the browser. It is quite different from that of most general-purpose programming platforms.
Stated formally, Node.js has a single-threaded, non-blocking, event-driven execution model. We will define each of these terms in this section.
Put simply, Node.js recognizes that many programmes spend most of their time waiting for other things to happen, for example, slow I/O operations such as disk access and network requests.
Node.js addresses this by making these operations non-blocking. This means that program execution can continue while they happen. For example, the filesystem API's stat function for retrieving statistics about a file may be called as follows:
fs.stat('/hello/world', function (error, stats) {
console.log('File last updated at: ' + stats.mtime);
});Two arguments are passed to the fs.stat function: the name of the file that we are interested in, and a callback function. The fs.stat call returns immediately, returning control of execution to the current thread but not returning a value. If there are further commands following the fs.stat call, these will then be executed. Otherwise, the thread is released to perform other work. The callback function is invoked (that is 'called back') only after the runtime has finished communicating with the filesystem. The result of the filesystem operation is passed into the callback function.
This non-blocking approach is also called
asynchronous programming. Other platforms support this (for example, C#'s async/await keywords and .NET's Task Parallel Library). However, it is baked in to Node.js in a way that makes it simple and natural to use. Asynchronous API methods are all called in the same way as fs.stat. They all take a callback function that gets passed error and result arguments.
The event-driven nature of Node.js describes how operations are scheduled. In typical procedural environments, a program has an entry point that executes a set of commands until completion, or enters a loop and performs some processing on each iteration.
Node.js has a built-in event loop, which isn't exposed to the developer. It is the job of the event loop to decide which piece of code to execute next. Typically, this will be a callback function that is ready to run in response to some other event. For example, a filesystem operation may have completed, a timeout may have expired, or a new network request may have arrived.
This built-in event loop simplifies asynchronous programming by providing a consistent approach and avoiding the need for applications to manage their own scheduling.
The single-threaded nature of Node.js simply means that there is only one thread of execution in each process. Also, each piece of code is guaranteed to run to completion without being interrupted by other operations. This greatly simplifies development and makes programs easier to reason about. It removes the possibility for a range of concurrency issues. For example, it is not necessary to synchronize/lock access to shared in-process state as it is in Java or .NET. A process can't deadlock itself or create race conditions within its own code. Single-threaded programming is only feasible if the thread never gets blocked waiting for long-running work to complete. Thus, this simplified programming model is made possible by the non-blocking nature of Node.js.
The built-in Node.js APIs provide a low-level core for creating applications. Applications typically only use a small number of these APIs directly. They often use third-party library modules that provide higher-level abstractions for application development.
Node.js has its own package manager, npm. This is similar to .NET's NuGet or the package management aspects of Java's Maven. Applications specify their dependencies in a simple JSON file.
The npm registry provides a central repository for packages. This registry has grown rapidly and is already much larger (in terms of number of available packages) than the corresponding repositories for other platforms (see http://www.modulecounts.com/). There are hundreds of thousands of packages available, providing a vast array of functionality.
The npm command line tool can be used to download packages and install new ones. Library dependencies are installed locally to each application. Some packages provide command-line tools, which may be installed globally rather than under a specific project.
Many frameworks available on npm are split into a small extensible core and a number of composable modules. This approach makes it easy to understand the libraries on which your application depends, avoiding the need to reason about complex heavyweight frameworks.
The consistency of calling non-blocking (asynchronous) API methods in Node.js carries through to its third-party libraries. This consistency makes it easy to build applications that are asynchronous throughout.
JavaScript is a language that can seem unintuitive compared to other popular object-oriented (OO) languages. It also has a number of quirks and flaws that have drawn criticism (and occasional ridicule). It might then seem a surprising choice of language for a new programming platform. This section discusses the factors that make JavaScript a more appealing choice.
The size and complexity of JavaScript is part of its appeal. The core language itself, which doesn't include APIs such as the DOM, is small and simple. This makes it easy for Node.js to establish its own styles and conventions.
The new APIs provided by Node.js and the consistent approach to asynchronous programming wouldn't be possible in a more complex language with a larger pre-existing standard class library.
JavaScript was first built as a programming language for client-side functionality in the browser. This might not make it an obvious choice for general-purpose programming.
In fact, these two use cases do have something important in common. User interface code is naturally event-driven (for example, binding event handlers to button clicks). Node.js makes this a virtue by applying an event-driven approach to general-purpose programming.
JavaScript supports functions as first-class objects. This means it's easy to create functions dynamically and pass around references to them. This fits in well with the asynchronous, non-blocking approach of Node.js. In particular, it's easy to expose and use APIs based around callback functions.
JavaScript has received a lot of attention in the last several years as it has become more widely used for providing rich functionality on the Web. Browser vendors have put a huge amount of engineering effort into improving the performance of JavaScript. Node.js benefits from this directly via its use of Chrome's V8 engine.
The JavaScript language itself is undergoing some major changes for the better. The ECMAScript 2015 standard (previously known as ES6) represents the most significant revision of the language in its history. It introduces features that make the language more intuitive and less verbose. It also addresses flaws that JavaScript has been criticized for in the past, removing gotchas and making programs easier to reason about.
As discussed earlier in this chapter, Node.js recognizes that I/O is a bottleneck for many applications. On most programming platforms, threads will waste time blocking on I/O operations. There are approaches developers can take to avoid this, but these all involve adding some complexity to their code. In Node.js, the platform itself provides a completely natural approach.
The flagship use case for Node.js is building web applications. These are inherently event-driven as most or all processing takes place in response to HTTP requests. Also, many websites do little computational heavy-lifting of their own. They tend to perform a lot of I/O operations:
Streaming requests from the client
Talking to a database, locally or over the network
Pulling in data from remote APIs over the network
Reading files from disk to send back to the client
These factors make I/O operations a likely bottleneck for web applications. The non-blocking programming model of Node.js allows web applications to make the most of a single thread. As soon as any of these I/O operations starts, the thread is immediately free to pick up and start processing another request. Processing of each request continues via asynchronous callbacks when I/O operations complete. The processing thread is only kicking off and linking together these operations, never waiting for them to complete. This allows Node.js to handle a much higher rate of requests per thread than other platforms. You can also still make use of multiple threads (for example, on multi-core CPUs) by simply running multiple instances of the Node.js process.
There are of course some applications that don't perform much I/O and are more likely to be CPU bound. Node.js would be less suitable for computationally-intensive applications. Programs that do a lot of processing of in-memory data are less concerned about I/O.
Web applications are not the only I/O-heavy applications though. Other classes of program that could be a good candidate for Node.js include the following:
Tools that manipulate large amounts of data on disk
Supervisor programs coordinating other software or hardware
Non-browser GUI applications that need to respond to user input
Node.js is especially suitable for glue applications that pull together functionality from other remote services. The increasing popularity of microservices as an architectural pattern makes this kind of application more common.
Node.js has been around for several years, but now is the perfect time to start using it if you haven't already.
The release of Node.js v4 towards the end of 2015 consolidated the project's governance model and heralds Node.js coming to maturity. It also allows the project to keep more up to date with the V8 engine. This means that Node.js can benefit more directly from ongoing development on V8. For example, security and performance improvements to V8 will now make their way into Node.js much sooner.
As discussed earlier in this chapter, the release of the ECMAScript 2015 standard makes JavaScript a much more appealing language. It pulls in useful features from other popular OO languages and resolves a number of long-standing flaws in JavaScript.
Meanwhile, the ecosystem of third party libraries and tools around Node.js and JavaScript continues to grow. Node.js is treated as a first-class citizen by major hosting platforms. Companies such as Google and Microsoft are also throwing their weight behind JavaScript and related technologies.
In this chapter, we have understood Node.js and its distinctive execution model, explored the growing ecosystem around Node.js and JavaScript, seen the reasons for JavaScript as a language choice, and described the kinds of application that can benefit from Node.js.
Now that you know how Node.js works and when to use it, it's time to dive in and get our first Node.js application up and running.
