What an exciting time to be a JavaScript developer! What was once only considered a language to add enhancements and widgets to a web page has since evolved into its own fully-fledged ecosystem. As of the beginning of 2015, it stands as the second most popular language in terms of questions tagged on stack overflow, next to only Java, with around a million questions tagged on it. There are tons of frameworks and environments to make it possible to run JavaScript almost anywhere. I believe Atwood's law says it best:
"Anything that can be written in JavaScript will eventually be written in JavaScript!"
While this quote dates back to 2007, it's never been more true than today. Not only can you use JavaScript to develop a complete single-page application such as Gmail, but you will also see how we can achieve the following projects with JavaScript in the coming chapters of the book:
With the introduction of Node.js, JavaScript has officially gone in a direction that was never even possible before. Now you can use JavaScript on the server and you can also use it to develop full-scale, enterprise-level applications. When you combine this with the power of MongoDB and its JSON-powered data, you can work with JavaScript in every layer of your application.
Let's quickly go through some basic concepts of Node.js and MongoDB, which would be helpful for you to follow the rest of the chapters in this book.
One of the most important things that people get confused about while getting introduced to Node.js is to understand what exactly it is. Is it a different language altogether, is it just a framework on top of it, or is it something else? Node.js is definitely not a new language, and it is not just a framework on JavaScript. It can be considered as a runtime environment for JavaScript built on top of Google's V8 engine. So, it provides us with a context where we can write JavaScript code on any platform where Node.js can be installed. Anywhere!
Now a bit about its history! Back in 2009, Ryan Dahl gave a presentation at JSConf that changed JavaScript forever. During his presentation, he introduced Node.js to the JavaScript community. After a roughly 45-minute talk, he concluded it, receiving a standing ovation from the audience in the process. He was inspired to write Node.js after he saw a simple file upload progress bar on Flickr, the image-sharing site. Realizing that the site was going about the whole process the wrong way, he decided that there had to be a better solution.
Now let's go through the features of Node.js, which makes it unique from other server-side programming languages.
The V8 engine was developed by Google and was open sourced in 2008. As we all know, JavaScript is an interpreted language and it will not be as efficient as a compiled language, as each line of code gets interpreted one by one while the code gets executed. The V8 engine brings in an efficient model here, where the JavaScript code will be compiled into machine-level code and the executions will happen on the compiled code instead of interpreting the JavaScript. But even though Node.js is using the V8 engine, Joyent, which is the company that is maintaining Node.js development, it does not always update the V8 engine to the latest versions that Google actively releases. This has led to the new branch named io.js, which we will discuss later in this chapter.
You might be asking, how does a single threaded model help? Typical PHP, ASP.NET, Ruby, or Java-based servers follow a model where each client request results in the instantiation of a new thread or even a process. When it comes to Node.js, requests are run on the same thread with even shared resources. A common question that is often asked is what will be the advantage of using such a model? To understand this, we should understand the problem that Node.js tries to resolve. It tries to do asynchronous processing on a single thread to provide more performance and scalability for applications that are supposed to handle too much web traffic. Imagine web applications that handle millions of concurrent requests; if the server makes a new thread for handling each request that comes in, it will consume a lot of resources and we would end up trying to add more and more servers to increase the scalability of the application. The single threaded asynchronous processing model has its advantage in the previous context, and you can process much more concurrent requests with less number of server-side resources. However, there is a downside to this approach; Node (by default) will not utilize the number of CPU cores available on the server it is running on without using extra modules like pm2.
Note
The point that Node.js is single threaded doesn't mean that it doesn't use threads internally. It is just that the developer and the execution context that the code has exposure to have no control over the threading model internally used by Node.js.
If you are new to the concept of threads and process, we would suggest you to go through some preliminary articles about these topics. There are plenty of YouTube videos as well on the same topic. The following reference could be used as a starting point:
http://www.cs.ucsb.edu/~rich/class/cs170/notes/IntroThreads/
One of the most powerful features of Node is that it is event-driven and asynchronous. So how does an asynchronous model work? Imagine you have a block of code and at some nth line you have an operation that is time consuming. So what happens to the lines that follow the nth line while this code gets executed? In normal synchronous programming models, the lines that follow the nth line will have to wait till the operation at that line completes. An asynchronous model handles this case differently. To handle this scenario in an asynchronous approach, we need to segment the code that follows the nth line into two sections. The first section is dependent on the result of the operation at the nth line and the second section is independent of the result.
We wrap the dependent code in a function with the result of the operation as its parameter, and register it as a callback to the operation on its success. Once the operation completes, the callback function will be triggered with its result. Meanwhile, we can continue executing the result-independent lines without waiting for the result. In this scenario, the execution is never blocked for a process to complete. It just goes on with callback functions registered on each ones completion. Simply put, you assign a callback function to an operation, and when Node determines that the completion event has been fired, it will execute your callback function at that moment.
We can look at an example to understand the asynchronous nature in detail:
console.log('One');
console.log('Two');
setTimeout(function() {
console.log('Three');
}, 2000);
console.log('Four');
console.log('Five');In a typical synchronous programming language, executing the preceding code will yield the following output:
One Two ... (2 second delay) ... Three Four Five
However, in an asynchronous approach, the following output is seen:
One Two Four Five ... (approx. 2 second delay) ... Three
The function that actually logs Three is known as a callback to the setTimeout function.
Writing applications with Node is really enjoyable when you realize the sheer wealth of information and tools at your disposal! Using Node's built-in package manager npm, you can find literally tens of thousands of modules that can be installed and used within your application with just a few keystrokes! One of the reasons for the biggest success of Node.js is npm, which is one of the best package managers out there with a very minute learning curve. If this is the first ever package manager that you are getting exposed to, then you should consider yourself lucky!
On a regular month, npm handles more than a billion downloads and it has around 150,000 packages currently available for you to download. You can view the library of available modules by visiting www.npmjs.com. Downloading and installing any module within your application is as simple as executing the npm install package command. Have you written a module that you want to share with the world? You can package it up using npm and upload it to the public www.npmjs.org registry just as easily! If you are not sure how a module you installed works, the source code is right there in your projects' node_modules/ folder waiting to be explored!!
While you develop web applications, you will always end up doing the validations for your UI both at the client and server sides, as the client-side validations are required for better UI experience and server-side validations are needed for better security of app. Think about two different languages in action: you will have the same logic implemented in both the server and client side. With Node.js, you can think of sharing the common function between server and client, reducing the code duplication to a large extent.
Ever worked on optimizing the load time for client-side components of your Single-Page Application (SPA) loaded from template engines such as underscore? You would end up thinking about a way we could share the rendering of templates in both server and client at the same time—some call it hybrid templating. Node.js resolves the context of duplication of client templates better than any other server-side technologies, just because we can use the same JS templating framework and the templates both at server and client.
If you are taking this point lightly, the problem it resolves is not just the issue of reusing validations or templates on the server and client. Think about an SPA being built; you will need to implement the subsets of server-side models in the client-side MV* framework also. Now, think about the templates, models, and controller subsets being shared on both client and server. We are solving a higher scenario of code redundancy.
Node.js is not just to write JavaScript in the server-side. Yes, we have discussed this point earlier. Node.js sets up the environment for the JavaScript code to work anywhere it can be installed. It can be a powerful solution to create command-line tools as well as fully-featured, locally run applications that have nothing to do with the Web or a browser. Grunt.js is a great example of a Node-powered command-line tool that many web developers use daily to automate everyday tasks such as build processes, compiling CoffeeScript, launching Node servers, running tests, and more.
In addition to command-line tools, Node is increasingly popular among the hardware crowd with the Node bots movement. Johnny-Five and Cylon.js are two popular Node libraries that exist to provide a framework to work with robotics. Just search on YouTube for Node robots and you will see a lot of examples. Also, there is a chance that you might be using a text editor developed on Node.js. GitHub's open source editor named Atom, which is hugely popular, is an example.
One of the important reasons behind the origin of Node.js was to support real-time web applications. Node.js has a couple of frameworks built for real-time web applications, which are hugely popular: Socket.io and sock.js. These frameworks make it very simple to build instant collaboration-based applications such as Google Drive and Mozilla's together.js. Before the introduction of WebSockets in the modern browsers, this was achieved via long polling, which was not a great solution for real-time experience. While WebSockets is a feature that is only supported in modern browsers, Socket.io acts as a framework, which also features seamless fallback implementations for legacy browsers.
In addition to the powerful non-blocking asynchronous nature of Node, it also has very robust networking and filesystem tools available via its core modules. With Node's networking modules, you can create server and client applications that accept network connections and communicate via streams and pipes.
As we mentioned earlier in this chapter, io.js is nothing but a fork of Node.js created to be updated with the latest development on both V8 and other developments in the JS community. Joyent was taking care of the releases in Node.js and the process that was followed in taking care of the release management of Node.js lacked an open governance model. It leads to scenarios where the newer developments in V8 as well as the JS community were not incorporated into its releases. For example, if you want to write JavaScript using the latest EcmaScript6 (ES6) features, you will have to run it in the harmony mode. Joyent is surely not to be blamed for this, as they were more concerned about stability of Node.js releases than frequent updates in the stack. This led to the io.js fork, which is kept up to date with the latest JavaScript and V8 updates. So, it's better to keep your eyes on the releases on both Node and io.js to be up to date with the Node.js world.
To see an example of just how lightweight Node can be, let's take a look at some sample code that starts up an HTTP server and sends Hello World to a browser:
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
}).listen(8080, 'localhost');
console.log('Server running at http://localhost:8080');A few basic lines of code are all it takes to write a complete Node application. Running it with a simple Node app.js command will launch an HTTP server that is listening on port 8080. Point any browser to http://localhost:8080, and you will see the simple output Hello World on your screen! While this sample app doesn't actually do anything useful, it should give you a glimpse of the kind of power you will have while writing web applications using Node.js. If you don't have the initial Node.js development environment set up, we will discuss it in the next chapter.
You should have heard of this proverb:
"If all you have is a hammer, everything looks like a nail!"
This makes a lot of sense in this context. Node.js is not a technology to depend on all the application problems that you intend to solve and if not chosen wisely, the decision will backfire. Node.js is well suited for applications that are expected to handle a huge amount of concurrent connections. Also, it should be noted that it is most suited for applications where each incoming request requires very few CPU cycles. This means that if you intend to do computation-intensive tasks on requests, it will end up blocking the event loop—thereby impacting other requests concurrently processed by the web server. Node.js is well suited for real-time web applications, such as chat rooms, collaboration tools, online games, and so on. So when deciding whether to use or not use Node.js, we should analyze the application context seriously and figure out whether Node.js really suits the context of the application.
Note
It is quite hard to debate over the use cases of Node.js in a detailed manner. However, the following stackoverflow thread does this so effectively and we strongly recommend you to go though the answers on this post if you are more interested in the use cases of Node.js: http://stackoverflow.com/questions/5062614/how-to-decide-when-to-use-node-js
As we have briefly gone through the features and concept of Node.js, now let's look into the NoSQL and MongoDB side.
Let's start by exploring the answers to the question, what exactly is a NoSQL database? NoSQL is a common term for database technologies that deviate from the traditional Relational Database Management System (RDBMS) concepts. The common reason for these database solutions to deviate from RDBMS database standards is to achieve and set a better standard of availability and partitioning capabilities than traditional RDBMS solutions.
To introduce you to this concept, we should have a look at the Brewer's theorem, which is otherwise known as the CAP theorem:
"It is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: Consistency, Availability, and Partition tolerance."
Traditional RDBMS solutions are good at consistency and compromise a little once it comes to providing better availability (data reads) and partitioning capabilities. Most of the NoSQL solutions have been evolved in this direction to achieve better heights in data availability and partitioning.
As this is a common term for any database technology that deviates from the concepts followed by RDBMS solutions such as MySQL, PostgreSQL, and so on, there are various subsets for NoSQL databases. Most popular subsets of NoSQL are document stores, key-value stores, and graph-based database solutions. MongoDB, which is the one we are going to try out, falls in the document store category. There are many more NoSQL solutions available in the market apart from MongoDB, such as Cassandra, Redis, Neo4j, Hbase, and so on.
As we discussed in the previous paragraph, MongoDB falls into the document store category of NoSQL databases. MongoDB is being actively developed by 10gen, which was later renamed to MongoDB I.inc. MongoDB is open source and its source is available on various platforms such as GitHub.
One of the most important reasons for the popularity of MongoDB is that it is a JSON-friendly database. It means that documents are stored and retrieved from MongoDB as JavaScript objects. Internally, this JSON data gets converted to BSON format while getting persisted to the system. So, this gives an extreme flexibility where we can use the same data format from client to server and eventually to the database.
A typical document (record) in a MongoDB collection (table) might look like the following code:
$ mongo
> db.contacts.find({email: 'jason@kroltech.com'}).pretty()
{
"email" : "jason@kroltech.com",
"phone" : "123-456-7890",
"gravatar" : "751e957d48e31841ff15d8fa0f1b0acf",
"_id" : ObjectId("52fad824392f58ac2452c992"),
"name" : {
"first" : "Jason",
"last" : "Krol"
},
"__v" : 0
}Another important feature of MongoDB is its schemaless nature. With a relational database, you are required to define (ahead of time) the exact structure of the data being stored, which is termed as the schema. This means that you must have defined the exact number of columns, length, and data type for every field in a table, and that each field must always match that exact set of criteria. Mongo provides a flexible nature where the documents that you store into the database need not follow any schema unless the developer enforces it through the application level. This makes MongoDB a great fit for Agile-based development, as you could carry out modifications on the application schema on fly.
Other than the JavaScript-friendly nature, one other resemblance of MongoDB with Node.js is that it is also designed with highly concurrent applications with heavy read operations in mind.
MongoDB also introduces the concept of sharding, which makes it possible to scale the database horizontally as well as vertically. If the application owner needs to increase the database capabilities, they could add up more machines into the stack. This is a cheaper option compared to investing on RAM of a single machine, which will be the case in RDBMS solutions.
All the advantages that we discussed come with some impact on the consistency, as MongoDB does not strictly adhere to the RDBMS standards like ACID transactions. Also, if you end up creating a data model that might need too many JOIN operations, then MongoDB won't make a good fit as it is not designed with too many aggregations even though the aggregations are possible via the MongoDB aggregation framework. MongoDB may or may not be the right solution for your application. You should truly weigh the pros and cons of each technology before making a decision to determine which technology is right for you.
Both Node and MongoDB are extremely popular and active in the development community. This is true for enterprises as well. Some of the biggest names in the Fortune 500 space have fully embraced Node to power their web applications. This is due in large part to the asynchronous nature of Node, which makes it a great alternative for high traffic, high I/O applications such as e-commerce websites and mobile applications.
Here's just a small list of some big companies that are working with Node:
PayPal
LinkedIn
eBay
Walmart
Yahoo!
Microsoft
Dow Jones
Uber
New York Times
MongoDB's use in the enterprise sector is equally as impressive and widespread, with an increasing number of companies adopting the leading NoSQL database server. Here's just a small list of some big companies that are working with MongoDB:
Cisco
Craigslist Inc.
Forbes
FourSquare
Intuit
McAfee
MTV
MetLife
Shutterfly
Under Armour
The remainder of this book is going to be a guided tour that walks you through creating a complete data-driven website. The website we create will feature almost every aspect of a typical large-scale web development project. The app will be developed using a popular Node.js framework called Express, and it will persist data using MongoDB. In the first few chapters, we will cover the groundwork involved in getting the core of the server up and serving content. This includes configuring your environment so you are up and running with Node and MongoDB, and a basic introduction to the core concepts of both technologies. Then, we will write a web server from scratch powered by ExpressJS, which will handle serving all of the necessary files for the website. From there, we will work with the Handlebars template engine to serve both static and dynamic HTML web pages. Diving deeper, we will make the application persistent by adding a data layer where the records for the website will be saved and retrieved via a MongoDB server.
We will cover writing a RESTful API so that other people can interact with your application. Finally, we will go into the details to see how to write and execute tests for all of your code. A summary is given in the following figure:
Wrapping up, we will take a brief detour as we examine some popular, merging frontend technologies that are becoming increasingly popular while writing SPAs. These technologies include Backbone.js, Angular, and Ember.js.
Last but not least, we will go into details about how to deploy your new website to the Internet using popular cloud-based hosting services such as Heroku and Amazon Web Services.
Tip
Downloading the example code
You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
In this chapter, we reviewed what is to be expected throughout the rest of this book. We discussed the amazing current state of JavaScript and how it can be used to power the full stack of a web application. Not that you needed any convincing in the first place, but I hope you're excited and ready to get started writing web applications using Node.js and MongoDB!
Next up, we will set up your development environment and get you up and running with Node, MongoDB, and npm as well as write and launch a quick Node app that uses MongoDB!
