What was once only considered a language to add enhancements and widgets to a web page has since evolved into its own fully-fledged ecosystem. As of the survey of year 2017 (https://insights.stackoverflow.com/survey/2017), it stands as the most popular language in terms of usage on stack overflow with around a million questions tagged on it. There are tons of frameworks and environments to make it possible to run JavaScript almost anywhere. I believe Atwood's law says it best:
"Any application that can be written in JavaScript will eventually be written in JavaScript!"
While this quote dates back to 2007, it's never been truer than today. Not only can you use JavaScript to develop a complete single-page application such as Gmail, but you will also see how we can use it to achieve the following projects in the coming chapters of the book:
- Completely power the backend using Node.js and Express.js
- Persist data with a powerful document oriented database such as MongoDB
- Write dynamic HTML pages using Handlebars.js
- Deploy your entire project to the cloud using services such as Heroku and Amazon Web Services (AWS)
With the introduction of Node.js, JavaScript has officially gone in a direction that was never even possible before. Now, you can use JavaScript on the server and you can also use it to develop full-scale, enterprise-level applications. When you combine this with the power of MongoDB and its JSON-powered data, you can work with JavaScript in every layer of your application.
Let's quickly go through some basic concepts of Node.js and MongoDB, which will help you in following the rest of the chapters in this book.
One of the most important things that people get confused about while getting introduced to Node.js is understanding what, exactly, it is. Is it a different language altogether, is it just a framework on top of it, or is it something else? Node.js is definitely not a new language, and it is not just a framework on JavaScript. It can be considered as a runtime environment for JavaScript built on top of Google's V8 engine. So, it provides us with a context where we can write JavaScript code on any platform where Node.js can be installed. Anywhere!
Now, a bit about its history! Back in 2009, Ryan Dahl gave a presentation at JSConf that changed JavaScript forever. During his presentation, he introduced Node.js to the JavaScript community. He concluded it after a roughly 45-minute talk, receiving a standing ovation from the audience in the process. He was inspired to write Node.js after he saw a simple file upload progress bar on Flickr, the image-sharing site. Realizing that the site was going about the whole process the wrong way, he decided that there had to be a better solution.
Now let's go through the features of Node.js, which make it unique from other server-side programming languages.
The V8 engine was developed by Google and was open sourced in 2008. As we all know, JavaScript is an interpreted language and it will not be as efficient as a compiled language, as each line of code gets interpreted one by one while the code gets executed. The V8 engine brings in an efficient model, where the JavaScript code is first interpreted and then compiled into machine-level code.
The new V8 5.9 provides a stable release that introduces TurboFan compiler which provides performance and mass optimization benefits. It also launches Ignition interpreter which is quiet efficient for all the small and big devices like servers or IOT devices etc that varies on memory spectrum. Due to such low memory footprint it delivers fast startup of an application. We can study benchmarks in following link : https://goo.gl/B15xB2
With two powerful updates, the v8 team is also working on Orinoco, which is a garbage collector that works on mechanism of parallel and concurrent compacting.
Such a high performance with promising results was the reason to push the node 8(LTS) launch date from may 2018 to october 2018. Currently we are using node 8 with a non-LTS version. It provides clean replace for users using node v4.x.x and above with no broken library. The version 8 also has various inbuilt features like buffer improvements and inbuilt promisify methods etc. We can study them in following link : https://goo.gl/kMySCS
With the advent of the web, the traditional JavaScript was intended to add simple functionality and minimal runtime in the browser. Thus, JavaScript was kept as a single-threaded scripting language. Now, just to get a brief idea regarding single-threaded model, let's consider the following diagram:
A single-threaded model creates a single Callstack in an execution context. In the preceding code, when the function getData() is invoked, the function is pushed in the stack for execution sequentially.
In the context of Node.js, JavaScript is the base scripting language, hence, Node.js is single-threaded. You might be asking, how does a single-threaded model help? Typical PHP, ASP.NET, Ruby, or Java-based servers follow a model where each client request results in the instantiation of a new thread or even a process.
When it comes to Node.js, requests are run on the same thread with shared resources. A question that is often asked is, what will be the advantage of using such a model? To understand this, we should understand the problem that Node.js tries to resolve. It tries to do asynchronous processing on a single thread to provide more performance and scalability for applications that are supposed to handle too much web traffic. Imagine web applications that handle millions of concurrent requests; if the server makes a new thread for handling each request that comes in, it will consume a lot of resources and we will end up trying to add more and more servers to increase the scalability of the application.
The single-threaded, asynchronous processing model has its advantage in the previous context, and you can process much more concurrent requests with fewer server-side resources. However, there is a downside to this approach; Node (by default) will not utilize the number of CPU cores available on the server it is running on, without using extra modules like pm2.
Note
The point that Node.js is single-threaded doesn't mean that it doesn't use threads internally. It is just that the developer and the execution context that the code has exposure to have no control over the threading model internally used by Node.js. If you are new to the concept of threads and process, I would suggest you go through some preliminary articles about these topics. There are plenty of YouTube videos on the same topic as well. The following reference could be used as a starting point:>http://www.cs.ucsb.edu/~rich/class/cs170/notes/IntroThreads/
One of the most powerful features of Node.js is that it is both event-driven and asynchronous. So, how does an asynchronous model work? Imagine you have a block of code and at some nth line you have an operation that is time consuming. What happens to the lines that follow the nth line while this code gets executed? In normal synchronous programming models, the lines that follow the nth line will have to wait until the operation at that line completes. An asynchronous model handles this case differently.
Let us visualize this scenario with the help of the following code and diagram:
In the preceding case, the setTimeout() method is provided by JavaScript (Node.js) API. Hence, this method is recognized as synchronous and is executed in a different execution context. According to functionality to setTimeout() , it executes the callback function after a specified duration, in our case after three seconds.
Further, the current execution is never blocked for a process to complete. When Node.js API determines that the completion of an event has been fired, it will execute your callback function at that moment.
In a typical synchronous programming language, executing the preceding code will yield the following output:
Writing applications with Node.js is really enjoyable when you realize the sheer wealth of information and tools at your disposal! Using Node.js's built-in package manager, npm, you can find literally tens of thousands of modules that can be installed and used within your application with just a few keystrokes! One of the biggest reasons for the success of Node.js is npm, which is one of the best package managers out there, with a very minute learning curve. If this is the first ever package manager that you are getting exposed to, you should consider yourself lucky!
On a regular month, npm handles more than a billion downloads, and it has around 150,000 packages currently available for you to download. You can view the library of available modules by visiting www.npmjs.com. Downloading and installing any module within your application is as simple as executing the following command:
npm install package Have you written a module that you want to share with the world? You can package it up using npm and upload it to the public registry at www.npmjs.org just as easily! If you are not sure how a module you installed works, the source code is right there in your project's node_modules/ folder waiting to be explored!
While you develop web applications, you will always end up doing the validations for your UI, both at the client and server sides, as the client-side validations are required for better UI experience and server-side validations are needed for better security of the app. Think about two different languages in action: you will have the same logic implemented in both the server and client sides. With Node.js, you can think of sharing the common function between server and client, reducing the code duplication to a large extent.
Ever worked on optimizing the load time for client-side components of your Single-Page Application (SPA) loaded from template engines such as Underscore? You would end up thinking about a way we could share the rendering of templates in both server and client at the same time; some call it hybrid templating.
Node.js resolves the context of duplication of client templates better than any other server-side technologies, just because we can use the same JS templating framework and the templates both at server and client.
If you are taking this point lightly, the problem it resolves is not just the issue of reusing validations or templates on the server and client. Think about an SPA being built; you will need to implement the subsets of server-side models in the client-side MV* framework also. Now, think about the templates, models, and controller subsets being shared on both client and server. We are solving a higher scenario of code redundancy.
Node.js is not just to write JavaScript in the server-side. Yes, we have discussed this point earlier. Node.js sets up the environment for the JavaScript code to work anywhere it can be installed. It can be a powerful solution to create command-line tools, as well as fully featured, locally run applications that have nothing to do with the web or a browser. Grunt.js is a great example of a Node-powered command-line tool that many web developers use daily to automate tasks such as build processes, compiling CoffeeScript, launching Node.js servers, running tests, and more.
In addition to command-line tools, Node.js is increasingly popular among the hardware crowd with the Node.js bots movement. Johnny-Five and Cylon.js are two popular Node.js libraries that exist to provide a framework to work with robotics. Just search on YouTube for Node.js robots and you will see a lot of examples. Also, there is a chance that you might be using a text editor developed on Node.js. GitHub's open source editor named Atom, which is hugely popular, is an example.
One of the important reasons behind the origin of Node.js was to support real-time web applications. Node.js has a couple of frameworks built for real-time web applications which are hugely popular: Socket.io and Sock.JS. These frameworks make it very simple to build instant, collaboration-based applications such as Google Drive and Mozilla's together.js. Before the introduction of WebSockets in the modern browsers, this was achieved via long polling, which was not a great solution for real-time experience. While WebSockets is a feature that is only supported in modern browsers, Socket.io acts as a framework, which also features seamless fallback implementations for legacy browsers.
Note
If you need to understand more on the use of WebSockets in applications, here's a good resource on MDN that you can explore:https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_client_applications.
In addition to the powerful non-blocking asynchronous nature of Node.js, it also has robust networking and filesystem tools available via its core modules. With Node.js's networking modules, you can create server and client applications that accept network connections and communicate via streams and pipes. Node contains a module named as fs or filesystem which is totally responsible for all kind of read write operations performed on files. It also takes an advantage of streaming feature of node to perform those operations.
Dividing the app with respect to a functionality unit is called a microservice. Each microservice becomes the self contained unit of deployment. Node.js is based on the common JS modules pattern which provides modularity in structure of an application. Such a pattern is used to create the microservices. With the increase of functionality, the number of microservices increases. To manage those services, the Node.js ecosystem provides powerful libraries like pm2. Therefore, it enables the elements of an application to be updated and scaled separately.
With the advent of Internet of things (IoT), the Node.JS ecosystem provides amazing library support for various devices like sensors, beacons, wearables, and so on. Node.js is considered as an ideal technology for managing the request made by those devices via its powerful backbone of streams and non-blocking I/O. Popular IoT board variants like Arduino, Raspberry Pi, and so on, have more than 300 Node.js packages. The developers building data-intensive, real-time applications often find Node.js as a natural fit.
To see an example of how lightweight Node.js can be, let's take a look at some sample code that starts up an HTTP server and sends Hello World to a browser:
var http = require('http');
http.createServer(function(req, res) {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('Hello World\n');
}).listen(8080, 'localhost');
console.log('Server running at http://localhost:8080'); A few basic lines of code are all it takes to write a complete Node.js application. Running it with a simple Node.js app.js command will launch an HTTP server that is listening on port 8080. Point any browser to http://localhost:8080, and you will see the simple output Hello World on your screen! While this sample app doesn't actually do anything useful, it should give you a glimpse of the kind of power you will have while writing web applications using Node.js. If you don't have the initial Node.js development environment set up, we will discuss it in the next chapter.
You may have heard of this proverb by an american psychologist, Abraham Maslow:
"If all you have is a hammer, everything looks like a nail!"
This makes a lot of sense in this context. Node.js is not a technology to depend for on all the application problems that you intend to solve, and if not chosen wisely, the decision to use it will backfire. Node.js is well suited for applications that are expected to handle a huge amount of concurrent connections. Also, it should be noted, it is most suited for applications where each incoming request requires very few CPU cycles. This means that if you intend to do computation-intensive tasks upon request, it will end up blocking the event loop, thereby impacting other requests concurrently processed by the web server. Node.js is well suited for real-time web applications, such as chat rooms, collaboration tools, online games, and so on. So, when deciding whether or not to use Node.js, we should analyze the application context seriously and figure out whether Node.js really suits the context of the application.
Note
It is quite hard to debate over the use cases of Node.js in a detailed manner. However, the following Stack Overflow thread does this effectively, and I strongly recommend you to go through the answers on this post if you are more interested in the use cases of Node.js: http://stackoverflow.com/questions/5062614/how-to-decide-when-to-use-node-js.
As we have briefly gone through the concept and features of Node.js, now let's look into the NoSQL and MongoDB side.
Let's start by exploring the answers to the question: What exactly is a NoSQL database? NoSQL is a common term for database technologies that deviate from the traditional Relational Database Management System (RDBMS) concepts. The common reason for these database solutions to deviate from RDBMS database standards is to achieve and set better standards of availability and partitioning capabilities than traditional RDBMS solutions.
To introduce you to this concept, we should have a look at the Brewer's theorem, which is otherwise known as the CAP theorem:
"It is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: Consistency, Availability, and Partition Tolerance."
Traditional RDBMS solutions are good at consistency, but compromise a little once it comes to providing better availability (data reads) and partitioning capabilities. Most of the NoSQL solutions have been evolved in this direction to achieve better heights in data availability and partitioning.
As this is a common term for any database technology that deviates from the concepts followed by RDBMS solutions such as MySQL, PostgreSQL, and so on, there are various subsets for NoSQL databases. Most popular subsets of NoSQL are document stores, key-value stores, and graph-based database solutions. MongoDB, which is the one we are going to try out, falls into the document store category. There are many more NoSQL solutions available in the market apart from MongoDB, such as Cassandra, Redis, Neo4j, HBase, and so on.
As we discussed in the previous paragraph, MongoDB falls into the document store category of NoSQL databases. MongoDB is being actively developed by 10gen, which has been renamed to MongoDB Inc. MongoDB is open source and its source is available on various platforms, such as GitHub.
We will look at the following various features of MongoDB:
- JSON-friendly database
- Schema-less design
- Various performance facets
One of the most important reasons for the popularity of MongoDB is that it is a JSON-friendly database. This means that documents are stored and retrieved from MongoDB as JavaScript objects. Internally, this JSON data gets converted to BSON format while getting persisted to the system. So, this gives extreme flexibility, where we can use the same data format from client to server and eventually to the database.
A typical document (record) in a MongoDB collection (table) might look like the following code:
$ mongo
> db.contacts.find({email: 'jason@kroltech.com'}).pretty()
{
"email" : "jason@kroltech.com",
"phone" : "123-456-7890",
"gravatar" : "751e957d48e31841ff15d8fa0f1b0acf",
"_id" : ObjectId("52fad824392f58ac2452c992"),
"name" : {
"first" : "Jason",
"last" : "Krol"
},
"__v" : 0
} After examining the preceding output, we can see a key called _id. It is a MongoDB ID which must be encoded as a binary JSON objectID(BSON). If failed to encode, MongoDB won't be able to retrieve or update an object.
Another important feature of MongoDB is its schema-less nature. With a relational database, you are required to define the exact structure of the data being stored ahead of time, which is termed as the schema. This means that you must have defined the exact number of columns, the length, and the data type for every field in a table, and that each field must always match that exact set of criteria. Mongo provides a flexible nature where the documents that you store into the database need not follow any schema unless the developer enforces it through the application level. This makes MongoDB a great fit for Agile-based development, as you could carry out modifications on the application schema on the fly.
Other than the JavaScript-friendly nature, one other resemblance between MongoDB and Node.js is that MongoDB is also designed with highly concurrent applications with heavy read operations in mind.
MongoDB also introduces the concept of sharding, which makes it possible to scale the database horizontally as well as vertically. If the application owner needs to increase the database capabilities, they could add more machines into the stack. This is a cheaper option compared to investing in the RAM of a single machine, which will be the case in RDBMS solutions.
The process of Indexing created a list of values called index for a chosen field. These indexes are used to query larger chunks of data. Using indexes accelerates the data retrieval speed and performance. The MongoDB client provides various methods like ensureIndex to create an index only if one doesn't exist.
Additionally, MongoDB has various commands to allow aggregation of data, such as group, count, and return distinct values.
All the advantages that we discussed come with some impact on the consistency, as MongoDB does not strictly adhere to the RDBMS standards like ACID transactions. Also, if you end up creating a data model that might need too many JOIN operations, then MongoDB won't make a good fit as it is not designed with too many aggregations, even though the aggregations are possible via the MongoDB aggregation framework. MongoDB may or may not be the right solution for your application. You should truly weigh the pros and cons of each technology before making a decision to determine which technology is right for you.
Both Node.js and MongoDB are extremely popular and active in the development community. This is true for enterprises as well. Some of the biggest names in the Fortune 500 space have fully embraced Node.js to power their web applications. This is due in large part to the asynchronous nature of Node.js, which makes it a great alternative for high traffic, high I/O applications such as e-commerce websites and mobile applications.
Here's just a small list of some big companies that are working with Node.js:
- PayPal
- eBay
- Walmart
- Yahoo!
- Microsoft
- Dow Jones
- Uber
- New York Times
MongoDB's use in the enterprise sector is equally as impressive and widespread, with an increasing number of companies adopting the leading NoSQL database server. Here's just a small list of some big companies that are working with MongoDB:
- Cisco
- Craigslist Inc.
- Forbes
- FourSquare
- Intuit
- McAfee
- MTV
- MetLife
- Shutterfly
- Under Armour
The remainder of this book is going to be a guided tour that walks you through the creation of a complete data-driven website. The website we create will feature almost every aspect of a typical large-scale web development project. The app will be developed using a popular Node.js framework called Express, and it will persist data using MongoDB. In the first few chapters, we will cover the groundwork involved in getting the core of the server up and serving content. This includes configuring your environment so you are up and running with Node.js and MongoDB, and a basic introduction to the core concepts of both technologies. Then, we will write a web server powered by ExpressJS from scratch, which will handle serving all of the necessary files for the website. From there, we will work with the Handlebars template engine to serve both static and dynamic HTML web pages. Diving deeper, we will make the application persistent by adding a data layer where the records for the website will be saved and retrieved via a MongoDB server.
We will cover writing a RESTful API so that other people can interact with your application. Finally, we will go into the details to see how to write and execute tests for all of your code. A summary is given in the following section.
Wrapping up, we will take a brief detour as we examine some popular, merging frontend technologies that are becoming increasingly popular while writing SPAs. These technologies include Backbone.js, Angular, and Ember.js.
Last but not least, we will go into detail about how to deploy your new website to the Internet using popular cloud-based hosting services such as Heroku and Amazon Web Services.
In this chapter, we reviewed what is to be expected throughout the rest of this book. We discussed the amazing current state of JavaScript and how it can be used to power the full stack of a web application. Not that you needed any convincing in the first place, but I hope you're excited and ready to get started writing web applications using Node.js and MongoDB!
Next up, we will set up your development environment and get you up and running with Node.js, MongoDB, and npm, as well as write and launch a quick Node.js app that uses MongoDB!
