We are going to take a look at the technology stack we'll be using throughout the book. After a brief discussion of Node, npm, MongoDB, and Express we will introduce Mongoose as an ODM (Object-Document Modeler), cover good and bad use cases, and introduce the two cornerstones of Mongoose.
By the end of this chapter you will have an understanding of the technology stack and where Mongoose fits in. We will also have set up the sample project that we will build throughout the book.
The benefits of Node are making it ever more popular, and a set of compatible technologies are fast becoming the basis of a new standard development stack.
For quite some time JavaScript was mainly thought of as a lightweight browser-based scripting language. Microsoft did support a JavaScript version of Classic ASP, but it was largely side-lined in favor of the VBScript version.
Fast-forward 10 years and there were some pretty impressive JavaScript-based apps on the Web, for example Gmail. The general view of JavaScript as a programming language was starting to change back then.
In 2010 a new server-side system called Node.js was starting to make waves in the Internet developer community. Node.js was a new way for using JavaScript on the server side again. And it was quick and efficient. It could make scaling a web application much more cost-effective by reducing the amount of hardware required per-site visitor or request.
By 2012 Node was one of the buzzwords of the start-up scene, and you can see why. Firstly, most Web developers have some JavaScript experience so it doesn't require you to learn a new language from scratch. This also means you can share some code between front-end and back-end, so you don't have to code the same thing twice in two different languages. An excellent example of this is form validation; you want real-time validation in the browser for a better user experience, but you also need to validate on the server side to protect your system. So you code the same thing twice. Using Node you can use the same validation script on the server side and the browser, so you only have to code it once, in JavaScript.
Second, there is the reduced cost of scaling, especially when dealing with large numbers of concurrent users. On the same hardware Node's efficiencies allow it to handle many more requests than classic stacks on Apache or IIS. That being said, adding scalability to Node is more complicated than other stacks. Unlike the others you can't just put it on a more powerful machine and set it running. By default Node will currently only run one process, using only one core of a machine. There are methods to address this issue, using a load balancer in front of several processes running alongside each other for example, and there are plans to enable future versions of Node to be able to manage this natively, directly addressing this issue.
The benefits of scalability do have a cost. The single process is a more complicated approach to server-side programming and requires a change in mindset.
Traditional stacks are generally multithreaded. This means that every new visitor or session is given a new thread, and these are never shared. One session's activity generally doesn't impact another, until the server resources are exhausted. For example, if Session 1 is doing a complex database write operation it may take a couple of seconds, but Session 2 continues running oblivious to this.
Node is single-threaded. This means that every visitor or session is added to that one thread. So it is possible for a two-second database write operation to hold up every other user for two seconds. Multiply this by just 10 users and you've got a big problem on your hands.
Addressing this requires a different way of coding.
In the traditional stack, the approach to coding would be one step after the other, as in the following steps:
First, take the data.
Then write this data to the database.
Send a confirmation message.
Wait for the next request.
This is blocking code, as you can only do one thing at a time. This is fine in the multithreaded stack, as you're only ever responding to one person's requests.
In the single-threaded stack, you may have to respond to several people's requests at the same time, so you can't afford to be stuck doing time-consuming operations or waiting for someone else to do something. To do this, the approach to coding becomes more like the following:
You give this data to someone.
They write this data to the database.
When they are done, they send a confirmation message; if this isn't something they can do, then they add it to your request list.
You're going to take the next request.
This is non-blocking code. You can only do one at a time. So you're getting someone else to do something for you, and telling them what to do when they have finished. You can then deal with the next request coming in without any delay.
The way to code this in JavaScript is to use callbacks. Most JavaScript coders start using them before they even know it, particularly anybody who uses libraries such as jQuery.
Take the basic jQuery document.ready method as shown in the following:
$(document).ready(function() {
console.log("document ready");
});This is an event driven callback. The $(document).ready() part is a method function of jQuery, and we are sending it a function function() that it can run at the appropriate time. So we are saying "Hi ready, here is what I want you to do once the document is ready, I'll leave it up to you to decide when that is". The callback function we are using in this example is the following code snippet:
function() {
console.log("document ready");
}The jQuery .ready() function is pretty complicated, so we're not going to look at that here. However, the construct is very useful to understand. Look at the following code snippet:
ready = function (callback) {
// do something
// do something else
// ....
// and so on
callback();
};So ready itself is a function, and this function accepts one parameter callback. The callback parameter is generally an anonymous function, like the one we looked at earlier. A very important point to note is that callback now exists in the scope of the ready function. This means that your callback function has access to any variables or objects created in the ready function.
Now consider the following standard Node "hello world" example:
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {
'Content-Type': 'text/plain'
});
res.write('Hello world');
res.end();
})
listen(8888, '127.0.0.1');Look familiar? This is sending a callback to the http.createServer method function. See how the parameters—req and res—are being sent to the callback function even though they haven't been defined or created anywhere. This works because the http.createServer function will create these objects before calling this callback function that will use them.
MongoDB has become the main database of choice for working with Node. Note that there are Node drivers for many other databases, including MySQL, Microsoft SQL Server, Reddis, PostgreSQL, CouchDB, and more.
MongoDB is popular as it is fast and flexible with excellent community support. It is a document-oriented database, fitting in somewhere between Key-Value stores and traditional relational databases. Despite being a document store, MongoDB also enables rich querying and secondary indexing of documents, setting it apart from other databases and making it a very powerful option.
MongoDB stores documents as BSON, which is effectively binary-encoded JSON. When you run a query you get a JSON object returned (or a string in JSON format, depending on the driver). Look at the following code snippet for example:
{ "_id" : ObjectId("4ffbc45c35097b5a1583ad71"), "firstname" : "Simon", "lastname" : "Holmes }So, a document is a set of keys (for example, firstname) and values (for example, Simon). The _id entry is a unique identifier that the underlying MongoDB driver will—by default—create for each new document.
If you are more experienced with relational databases, it may help you to think of a document as a bit like a row in a table. In this analogy, the key can be thought of as a column. An important difference is that each document doesn't have to contain the exact same set of keys, and there is no direct need to have keys with empty values taking up space.
A collection of documents is called a collection. The closest analogy is a table. So in your database you could well have multiple collections, such as a users collection, posts collection, and stats collection.
MongoDB is also extremely scalable, with many built-in capabilities for distributing across multiple servers, without compromising speed or data integrity.
With everything combined, it makes MongoDB a great playmate for Node.
Express is a web application framework for Node.
When you create a Node project, you have to do a lot more groundwork than you might be used to. Until you create it, there isn't even a web server. Then you have to deal with serving static files in a non-blocking way, figure out the routing mechanism, view engine, session management, and so on.
Or you can create an Express project and let it do all of this for you, in a tried-and-tested way. At the end of this chapter, we'll see how easy it is to set up an Express project.
Note that Express is not required to build a Node application, but it is a great starting point for building web applications.
Mongoose is an object modeling tool for MongoDB and Node.js. What this means in practical terms is that you can define your data model in just one place, in your code.
Yes, that's right. You don't have to create a schema in the database, link that to an ORM or map it into your project objects and classes. You can just define your data structure in JSON inside your project.
The first time I created a project like this I was amazed at how much time and frustration it saves. Even now I still get that warm glow when I start a new project or prototype using Mongoose. It's like taking a shortcut to work down deserted country lanes while everybody else is gridlocked on the highway.
A schema definition can be as simple as the following code snippet:
var userSchema = new mongoose.Schema({
firstname: String,
lastname: String,
createdOn: Date
});A document in MongoDB created from this schema would be like the following code snippet:
{ "__v" : 0, "_id" : ObjectId("51412597e8e6d3e35c000001"),"createdOn" : ISODate("2013-03-14T01:19:19.866Z"),"firstname" : "Simon", " lastname " : "Holmes" }If you want to refactor, then you can just do it from within your code, saving a huge amount of development time.
Mongoose is primarily useful when you want to interact with structured data in MongoDB. It allows you to define a schema for your data, so that you can interact with your MongoDB data in a structured and repeatable way.
Mongoose helps with many common MongoDB tasks, and removes some of levels of complexity from the nested callbacks you find yourself getting lost in with the native MongoDB driver.
Mongoose also returns the data to you as a JSON object that you can use directly, rather than the JSON string returned by MongoDB.
Mongoose also has a whole suite of helper functions and methods that we'll explore throughout the subsequent chapters of this book.
Mongoose is probably not the answer for you if you are primarily working with the following:
Schema-less data
Random documents
Pure Key-Value pairs
There are two aspects of Mongoose that we need to introduce you to before going much further:
Schemas
Models
This is a very high-level view; so don't worry if you're not 100 percent confident with this just yet, as we'll be covering it in a lot more detail later in this book.
As we saw earlier, a schema is fundamentally describing the data construct of a document. This schema defines the name of each item of data, and the type of data, whether it is a string, number, date, Boolean, and so on.
var userSchema = new mongoose.Schema({
name: String,
email: String,
createdOn: Date,
verified: Boolean
});In most scenarios you would have one schema for each collection within the database.
Schemas are a powerful aspect of Mongoose, which can also be extended with helper functions and additional methods. But we'll describe more about that in a later chapter.
A model is a compiled version of the schema. One instance of the model will map to one document in the database.
Creating a User instance based on the schema userSchema is a one line task:
var User = mongoose.model('User', userSchema);It is the model that handles the reading, creating, updating, and deleting of documents.
Now that we know a little bit about the technology, it's time to get everything installed. What follows is not an in-depth installation tutorial, but covers the necessary steps to get everything installed on Ubuntu. The installation process is very similar on other Linux distributions and Mac OS X. For Windows users, Node and MongoDB have installation packages you can run, and after that everything else is pretty much the same.
Node needs a couple of things installed to run properly, Python and a C compiler. So let's make sure those are installed and up-to-date with a terminal command. I use the Ubuntu Advanced Packaging Tool (APT) to install most software, as it makes it this easy:
$ sudo apt-get install python-software-properties python g++ make
Again we'll use APT to install Node.js. As Node is actively developed and frequently updated, APT doesn't always have the latest build. The latest version on APT is currently several versions behind, so we will add the repository of one of the Node.js developers. This is also the approach recommended on the Node website nodejs.org.
$ sudo add-apt-repository ppa:chris-lea/node.js $ sudo apt-get update $ sudo apt-get install nodejs
If everything has gone smoothly then you should be able to check the installed version displayed in terminal using the following command:
$ node --version
You should be greeted with a message back in the terminal along the lines of v0.10.0.
npm is a package manager for Node, although npm does not actually stand for Node Package Manager. As of Node Version 0.6.3, npm is actually automatically installed when you install Node.
You can quickly check that this has been done by checking the version number on the terminal.
$ npm --version 1.1.23
If it is not there for some reason, simply install it with APT with the following command:
$ sudo apt-get install npm
$ sudo apt-get install mongodb
In order to test MongoDB, we need to run it. You can do this by just typing mongodb into terminal, but I prefer to run it as a persistent service so that you don't have to restart it.
$ sudo service mongodb start
This should give you a confirmation message in terminal that the mongodb process is running.
Now to finally test that it has installed correctly we'll drop into the built-in MongoDB shell. In terminal, enter the following:
$ mongo
This should do three things:
Enter you into the MongoDB shell
Show you the version of the shell being used
Connect to the
testdatabase that comes with MongoDB
You should see something like the following screenshot:
Now we're starting to get into the Node-based applications. So those of you who are working on different operating systems can start paying attention again now. We are going to install Express using npm.
$ sudo npm install -g express
Note
See the -g flag there in the command line? This installs express globally, meaning that we can use it wherever we want.
Once again you can test the installation by checking the version number.
$ express --version
Finally we get round to installing Mongoose!
There are two ways of installing Mongoose, both using npm. I recommend using the latter one.
If you're creating a quick test app, or will never have to move or publish the project elsewhere, you can use npm to install Mongoose directly into your project. Navigate to the root folder of your site or project in terminal and run the following command:
$ sudo npm install mongoose
Any Node project can have a package.json file, and any packages you install via npm will have one. This file is normally in the root of your project, and can contain various metadata, such as project name, description, version number, and authors. It also helps manage the project's dependencies.
If you ever need to move, copy, or distribute your project you will find this invaluable. The dependency management means that you are not forced to remember which versions of which modules you added to each particular project. It becomes even more necessary if you distribute it widely, as it removes any doubt from the end-user's mind as to what the project needed.
Before we get into creating one, the following code snippet is an example of a package.json file following code snippet that you might expect to see when you create a new project using Express.
{
"name": "application-name",
"version": "0.0.1",
"private": true,
"scripts": {
"start": "node app"
},
"dependencies": {
"express": "3.0.0rc5",
"jade": "*"
}
}To install a specific version of Mongoose—which is the recommended approach, especially if you are going to distribute your project—you would update the dependencies section to look like the following code snippet:
"dependencies": {
"express": "3.0.0rc5",
"jade": "*",
"mongoose": "3.6"
}To install or update all of the dependencies listed in a package.json file, using terminal, you simply navigate to the folder containing the file and run the following command:
$ npm install
Now that we have the entire building blocks ready, let's set the foundations. Throughout this book, we are going to build a simple project management web app, called MongoosePM.
Let's start off by creating an Express project.
In terminal, navigate to a relevant folder, for example,
/MyWebsitesin Linux, orC:\Users\Your Username\Documents\My Web Sites\in Windows.Enter the following command:
$ express --sessions mongoosepm
This will create a new folder called mongoosepm. Inside this folder it will create a new Express project. By default an Express project doesn't support user sessions; an Express project is essentially stateless. By adding the --sessions parameter in the command line we add session support to our project, so that we can follow a user from page to page without having to constantly re-authenticate.
After Express has finished doing its thing you'll be able to see a whole load of new files and folders within your mongoosepm folder. The intricacies of Express are beyond the scope of this book, which is one of the reasons we have mainly stuck with the default settings.
The next step is normally to install the dependencies, but first let's add Mongoose and Connect to the dependency list. So open up the package.json file in the root of the project and add a couple of lines to the dependencies hash.
For example, if your dependencies section looks like the following code snippet:
"dependencies": {
"express": "3.0.0rc5",
"jade": "*"
}Then you change it with this (see the following code):
"dependencies": {
"express": "3.0.0rc5",
"jade": "*",
"connect": "2.7.x",
"mongoose": "3.6.x"
}It is recommended that you don't "wildcard" the version for most modules, as there is no guarantee that code based on older versions will work on newer versions. Putting a wildcard in for the patch version like we have for Connect and Mongoose, is recommended as they should only be patch updates and be backwards compatible.
To install these dependencies, go to the mongoosepm folder in terminal and run the following command:
$ npm install
Once this has finished working, we should have a fully functional Express website to play with. Let's test it to make sure!
Run the following command from terminal:
$ node app
You should see the following confirmation message directly in terminal:
Express server listening on port 3000
So open up a web browser, and point it to http://localhost:3000. All being well, you should be presented with the default Express project landing page, as shown in the following screenshot:
Tip
Downloading the example code
You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.
In this chapter we have taken a whirlwind tour of the technology stack, and installed the required programs, which are as follows:
Node.js
npm
MongoDB
Express
Mongoose
You should have at least a basic understanding of the majority of the stack. We're not going to be discussing in depth the workings of Node, MongoDB, and Express, but I will make sure to briefly describe any new concepts as we go.
In this first chapter, we have also created the framework for the project that we are going to build in later chapters. This is a website built on Node.js, using the Express framework. We have tested and seen this working in a browser.
In the next chapter, we will take a look at how to create Mongoose connections to database, and the best way to add a connection to our new project.
