Home Web Development Mastering Web Application Development with Express

Mastering Web Application Development with Express

By Alexandru Vladutu
books-svg-icon Book
eBook $32.99 $22.99
Print $54.99
Subscription $15.99 $10 p/m for three months
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
eBook $32.99 $22.99
Print $54.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Diving into Express
About this book
Publication date:
September 2014
Publisher
Packt
Pages
358
ISBN
9781783981083

 

Chapter 1. Diving into Express

Express is the de facto web application framework for Node.js and one of the most depended-upon modules, according to the NPM registry.

In this chapter, we will cover the following topics:

  • The main features of the framework

  • The comparison of Express with other web application frameworks

  • Using the right tool for the right job

  • The important companies that use Express in production

  • How to structure applications with Express

 

The best parts of Express


When searching the Web for information on Express, we find that it is a minimal and flexible web framework that adds the essential bits and pieces needed to create powerful web applications.

It is minimal because it provides the basic features we need to create web applications, such as routing based on URL paths (it has DSL to describe routes), support for template engines, cookie and session management, the parsing of incoming requests, and so on. Without these built-in features, we need to create our own custom solutions on top of the Node HTTP. The source code for Express is just a few thousand lines of code, enabling us to easily dig deeper for a better understanding of how things work internally.

The flexibility comes from the fact that this framework does not impose things such as a certain application structure or database layer. Furthermore, not every middleware available is included by default when creating an application (unlike other big, monolithic frameworks); we have to explicitly include what we want. Even though Express is not a typical Model-View-Controller (MVC) framework, there's nothing stopping us from customizing it to be one if our requirements dictate it.

We can build different kinds of applications with Express, such as REST APIs, single-page and multipage applications, real-time applications, applications that spawn external processes and output their result, and many others. Due to its intuitive API and flexibility, Express makes it easy for newcomers to get started with the framework and use it for rapid prototyping when needed. Although there are methods to facilitate certain actions (such as redirecting the user to another page or serving JSON data), the functions built into Node are also available for this purpose.

The out-of-the-box performance of Express is really good; it can handle thousands of concurrent connections per second (the results are dependent on the concrete use case). An application can always be improved through caching, scaling to multiple processes, or other techniques, but it's good to know that Express won't be our bottleneck.

 

Comparing Express with other frameworks


When comparing a web framework to another, we first need to ask ourselves what problems each framework is trying to solve. After that, we can move on to compare their functionality and choose the one that suits our projects best.

Goal

Express was built to help developers with HTTP, not to be a full-stack framework that's packed with features. The framework gives us all the primitives to create all kinds of applications, from simple web applications to hybrid and real-time ones. Unlike big, monolithic frameworks, Express is not packed with things such as ORMs, view helpers, or other complex features. This means that we have the flexibility to plug in whatever we want to.

Conventions

When starting out with opinionated frameworks such as Rails, we need to learn about their conventions; a few examples of what we need to know are as follows:

  • Where things go inside the application folder

  • The naming conventions

  • How to define data relationships

These conventions can be an advantage for teams with many developers (to keep everybody on the same page), but if we need to create smaller applications or want to avoid the steep learning curve, Express is a better option.

The fact that Express isn't opinionated can be viewed as a good thing or a disadvantage depending on the use case. It's flexible enough that we can create our own conventions, but at the same time, we might not want or have time to do that.

Databases

Some frameworks are tied into a particular database or Object Relational Mapper (ORM), but that isn't the case with Express. It doesn't care about how we manage our data, so it doesn't tie us to a database, nor does it include drivers for any.

If we decide to add a database or an ORM to our application, we need to manually include it.

Views

There are a lot of templating engines available for Express, and it's very simple to integrate new ones. Some of them handle layouts and partials so we can reuse code and provide other features.

Express has support for view helpers, but the framework doesn't provide any out-of-the-box support.

Overall

Express is a good choice if we want as much control over our applications as possible, without having to recreate basic, HTTP-related functionality over and over again. It adds the bare minimum sugar syntax to create web applications and doesn't force us into using a certain database, ORM, or templating engine.

Since it's a minimalist framework, we can't expect it to have as many features as the more complex frameworks such as Rails, Django, or CakePHP.

 

Use cases


Before diving into the code, we need to consider whether Express is a good choice for the application we need to create. Next, we will check out a couple of good use cases for the framework.

Complex applications with heavy I/O bound operations

The Web is constantly evolving, and nowadays, applications do more than talk to a single database and send HTML over the wire. An application could use an in-memory database for session storage and caching, a message queue for background processing, at least one relational/NoSQL database, and external services to store files, stream logs, and monitor application health. The handling of I/O bound operations is a great use case for Node because of its nonblocking nature, and this applies to Express as well. This means that we can easily integrate all these components into our project, and it will still have a solid performance.

Single-page applications

Single-page applications represent web applications that don't reload the page when we use them. They update parts of their interface to provide a more native-like experience to end users.

There are many arguments for writing single-page applications in Express, which include the following:

  • It has the ability to handle a lot of concurrent requests per second

  • It's not a bloated framework; it has the bare minimum glue needed to write web applications

  • It has a lovely DSL syntax to describe routes

  • It can perform content negotiation, so we can have the same endpoint for our data but deliver different representations based on the client's request (JSON, XML, or others)

  • It has a lot of small functions that make our lives easier, such as res.sendfile, which transfers a file to the client and sets the proper headers, and req.xhr, which checks whether the current request has been transmitted using Ajax, and many others

Reusable applications

Any Express application can be mounted onto another parent one, enabling us to create modular components that can be included in multiple applications. For example, we can create a user authentication system and reuse it for all our projects. Another situation where this can come in handy is when multiple people are working on the same project but each has different responsibilities; somebody could work on an API while a different person creates the multipage website, and each of them could use separate repositories for source control management. When a child application is finished, the master one can include it by adding a single line of code (okay, maybe two if we're declaring the first one as a dependency in the package.json file).

Code sharing between the server and the client

Writing application code that runs both on the server and on the client is a very hot topic right now, and has often been referred to as "The Holy Grail of Web Development". Besides eliminating code duplication, there is one other big advantage of using this approach with single-page applications: we can render the first page on the server and all the subsequent ones on the client. This improves the speed of the initial page load (so users don't have to wait until all the JavaScript code is loaded and executed) and is also SEO friendly and doesn't require us to resort to tricks such as proxying to a headless browser for crawlers. There have been several attempts to create frameworks that use the Backbone.js client-side API on top of Express, one of the most popular being Rendr (https://github.com/airbnb/rendr).

A base to create more complex frameworks

Since Express is a minimalist framework, we can use it to build more complex and opinionated solutions. In fact, there are lots of such MVC and real-time frameworks in the Node ecosystem. They offer advanced features that are packed into a single application, such as an ORM, middleware that boosts security, internationalization/localization, application life cycle middleware, a custom routing library, built-in view helpers, and so on. Another aspect to take into consideration is that these frameworks also impose certain conventions that we need to adhere to, the most notable being the application structure.

Bad use cases

If there is any CPU-intensive task that is blocking the event-loop, it means that every single client making a request to the Express application will just hang until that task has finished. This happens because, unlike the Apache web server that spawns a thread per connection, Node uses an event loop, so all the requests run in the same thread.

If we want to create a regular CRUD-based application that has complex database relationships—and scaling thousands of concurrent requests isn't the primary goal—then using a full-stack framework is a better option (for example, Rails, Django, and CakePHP). That's not to say that we cannot achieve the same end result with Express, but we would have to include all the components ourselves.

 

Express into the wild


Whether we are trying to introduce a new tool into a technology stack at our company or simply want to experiment with new stuff once in a while, we need to ask ourselves the following questions before diving straight in:

  • Is it still an active project or has it been abandoned?

  • Is it mature enough or do I have to battle-test it myself?

  • Which companies are using it in production?

Express is the most popular web framework for Node, with more than a hundred contributors and thousands of commits, the first commit dating back to June 2009. Its repository is one of the most watched on GitHub. These facts answer the first two questions, so next, we'll talk about who is using it in production.

Popular companies such as MySpace, eBay, Uber, and Mozilla use Express in production, and others have made their own framework/project on top of it; here's a list of them:

  • Yahoo! created an MVC framework called Mojito that can run on both the client side and server side

  • PayPal released Kraken.js, an opinionated kind of Express with support for localization, application security, environment-based configuration, and other features baked in

  • Airbnb's Rendr library allows us to run Backbone.js both on the client and on the server

  • Ghost is a popular open source blogging platform with an elegant UI that can be used either as a standalone or by being attached to an existing Express application

  • Sails.js is a real-time MVC framework based on Express and Socket.IO that has a lot of advanced features, such as automatic JSON API generation, role-based access control, and a database agnostic ORM

  • Compound.js is an MVC framework that highly resembles Rails: it has scaffolding, a similar application structure, a lot of custom helpers, an ORM with relations support, and built-in validation as well as other useful features

 

The application structure


One of the most frequently asked questions by newcomers to Express is how to structure an application. There is no definitive answer for this, and we may choose different solutions based on how big our application is or what problem we are trying to tackle. Luckily for us, Express is easy to customize, and we can apply whatever structure we deem necessary.

If the code base is small, we can include everything into a few files or even a single one. This might be the case when exposing a database over HTTP (such as LevelDB and PouchDB) and creating mountable applications (these tend to be small and solve a specific problem) or other small applications.

When dealing with medium and large projects, the best thing to do is to split them into smaller pieces, making them easier to debug and test. If there are parts of the application that can be reused for other projects, the best thing to do is to move them into their separate repository.

Group files by features

An interesting technique to structure an application is to group files by the features they provide instead of grouping them by their function. In MVC, the controllers, models, and views live inside their own folder; however, with this approach, we have folders that group files with the same role. For example, consider the following folders:

  • Signup: This includes the route handler for the signup process and its view

  • Login: This is similar to the signup feature

  • Users: This contains the model for the users so that it can be shared between different features

  • posts-api: This exposes a RESTful interface for the articles of the site and contains the routes and model of the posts

One could go even further and choose to include things such as tests and static assets that belong to a feature inside its folder.

If there's something that can be reused for multiple features such as the general layout or models, we can group them inside their own folder. Each of these folders can export an Express application with its own view engine, middleware, and other customizations. These folders can reside in a parent lib folder, for example. We will then require them in the main app.js file like we would any regular middleware. It's a good way to separate concerns, although they are not necessarily complete, independent pieces because they rely on application-specific logic.

An advantage this structure offers is that when we are working on a certain section of an application, all the files that need to be created/edited are in the same location, so there's no need to switch between controllers, models, and views like with MVC.

It's worth mentioning that the creator of Express, TJ Holowaychuk, recommends this approach for larger applications instead of MVC.

Model-View-Controller

The most common technique to structure web applications with Express is MVC. When generating a project using the Express CLI, it almost provides an MVC structure, omitting the models folder. The following screenshot lists all the files and folders generated for a sample application using the CLI tool:

The package.json file is automatically populated with the name of the application, the dependencies, the private attribute, and the starting script. This starting script is named app.js and loads all the middleware, assigns the route handlers, and starts the server. There are three folders in the root:

  • public: This folder contains the static assets

  • views: This folder is populated with Jade templates by default

  • routes: This folder includes the routes (these are the equivalent controllers)

Apart from these existing folders and the models folder, which we need to create ourselves, we might also create folders for tests, logs, or configuration. The best thing about this structure is that it's easy to get started with and is known to most developers.

Developing a real MVC application

Let's apply the theory in practice now and create an MVC file manager application using Express 4.x and Mongoose (an object modeling library for MongoDB). The application should allow users to register and log in and enable them to view, upload, and delete their files.

Bootstrapping a folder structure

We will start by creating the folder structure. First, we'll use the Express CLI tool in the terminal to create the boilerplate. Apart from the public, routes, and views folders, we also need to add folders for models, helpers (view helpers), files (the files uploaded by users will be stored in subfolders here), and lib (used for internal app libraries):

$ express FileManager
$ cd FileManager
$ mkdir {models,helpers,files,lib}
Installing NPM dependencies

By default, the CLI tool will create two dependencies in your package.json file—express and jade—but it won't install them, so we need to manually execute the following install command:

$ npm install .

In addition to these two modules, we also need to install mongoose to interact with MongoDB, async for control flow, pwd to hash and compare passwords, connect-flash to store messages for the user (which are then cleared after being displayed), and connect-multiparty to handle file uploads. We can use the following shortcut to install the packages and have them declared in package.json at the same time if we call NPM with the –save flag:

$ npm install –save mongoose async pwd connect-flash connect-multiparty

Express 3.x came bundled with the Connect middleware, but that's not the case in the 4.x version, so we need to install them separately using the following command:

$ npm install –save morgan cookie-parser cookie-session body-parser method-override errorhandler

Note

The middleware libraries from Connect were extracted into their separate repos, so starting with Express 4.x, we need to install them separately. Read more about this topic on the Connect GitHub page at https://github.com/senchalabs/connect#middleware.

We can always check what modules are installed by entering the following command in the terminal at the root of our project:

$ npm ls

That command will output a tree with the dependencies.

Note

It's worth noting that the versions for the dependencies listed in the package.json file will not be exact when we use the –save flag; instead, they will be using the default npm semver range operator. You can read more from the official npm documentation (https://www.npmjs.org/doc/cli/npm-install.html) and the node-semver page (https://www.npmjs.org/package/semver).

Setting up the configuration file

We can get as inventive as we want with the configuration parameters of a project, like have multiple subfolders based on the environment or hierarchical configuration, but for this simple application, it's enough to have a single config.json file. The configuration variables we need to define in this file are the MongoDB database URL, the application port, the session secret key, and its maximum age so that our file will look like the following code:

{
  "mongoUrl": "mongodb://localhost/filestore",
  "port": 3000,
  "sessionSecret": "random chars here",
  "sessionMaxAge": 3600000
}

Tip

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

You can also download the example code files for the book from GitHub: https://github.com/alessioalex/mastering_express_code.

The starting script

In the main file of the application, named app.js, we handle the view setup, load the middleware required for the project, connect to the database, and bind the Express application to a port. Later on, we modify this file to set up the route handling as well, but at the moment, the file contains the following code:

// Module dependencies
var express = require('express');
var app = express();
var morgan = require('morgan');
var flash = require('connect-flash');
var multiparty = require('connect-multiparty');
var cookieParser = require('cookie-parser');
var cookieSession = require('cookie-session');
var bodyParser = require('body-parser');
var methodOverride = require('method-override');
var errorHandler = require('errorhandler');
var config = require('./config.json');
var routes = require('./routes');
var db = require('./lib/db');

// View setup
app.set('view engine', 'jade');
app.set('views', __dirname + '/views');
app.locals = require('./helpers/index');

// Loading middleware
app.use(morgan('dev'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(methodOverride(function(req, res){
  if (req.body && typeof req.body === 'object' && '_method' in req.body) {
    // look in url - encoded POST bodies and delete it
    var method = req.body._method;
    delete req.body._method;
    return method;
  }
}));
app.use(cookieParser());
app.use(cookieSession({
  secret: config.sessionSecret,
  cookie: {
    maxAge: config.sessionMaxAge
  }
}));
app.use(flash());

if (app.get('env') === 'development') {
  app.use(errorHandler());
}

// static middleware after the routes
app.use(express.static(__dirname + '/public'));

// Establishing database connection and binding application to specified port
db.connect();
app.listen(config.port);
console.log('listening on port %s', config.port);
The database library

Note that the preceding app.js file contains the code for the database connection. Later on, we will need other database-related functions such as checking for failed data validation, duplicate keys, or other specific errors. We can group this logic into a separate file called db.js inside the lib folder and move the connection functionality there as well, as shown in the following code:

var mongoose = require('mongoose');
var config = require('../config.json');

exports.isValidationError = function(err) {
  return ((err.name === 'ValidationError')
          || (err.message.indexOf('ValidationError') !== -1));
};

exports.isDuplicateKeyError = function(err) {
  return (err.message.indexOf('duplicate key') !== -1);
};

exports.connect = /* database connection function extracted from app.js should move here */
Routes

The routes folder will have a file for each controller (files.js, users.js, and sessions.js), another file for the application controller (main.js), and an index.js file that will export an object with the controllers as properties, so we don't have to require every single route in app.js.

The users.js file contains two functions: one to display the user registration page and another to create a user and its subfolder inside /files, as shown in the following code:

var User = require('../models/user');
var File = require('../models/file');
var db = require('../lib/db');

exports.new = function(req, res, next) {
  res.render('users/new', {
    error: req.flash('error')[0]
  });
};

exports.create = function(req, res, next) {
  var user = new User({ username: req.body.username });

  user.saveWithPassword(req.body.password, function(err) {
    if (err) {
      if (db.isValidationError(err)) {
        req.flash('error', 'Invalid username/password');
        return res.redirect('/users/new');
      } else if (db.isDuplicateKeyError(err)) {
        req.flash('error', 'Username already exists');
        return res.redirect('/users/new');
      } else {
        return next(err);
      }
    }

    File.createFolder(user._id, function(err) {
      if (err) { return next(err); }

      req.flash('info', 'Username created, you can now log in!');
      res.redirect('/sessions/new');
    });
  });
};

The sessions.js file handles user authentication and sign out as well as renders the login page. When the user logs in successfully, the username and userId properties are populated on the session object and deleted on sign out:

var User = require('../models/user');

exports.new = function(req, res, next) {
  res.render('sessions/new', {
    info: req.flash('info')[0],
    error: req.flash('error')[0]
  });
};

exports.create = function(req, res, next) {
  User.authenticate(req.body.username, req.body.password, function(err, userData) {
    if (err) { return next(err); }

    if (userData !== false) {
      req.session.username = userData.username;
      req.session.userId = userData._id;
      res.redirect('/');
    } else {
      req.flash('error', 'Bad username/password');
      res.redirect('/sessions/new');
    }
  });
};

exports.destroy = function(req, res, next) {
  delete req.session.username;
  delete req.session.userId;
  req.flash('info', 'You have successfully logged out');
  res.redirect('/sessions/new');
};

The files.js controller performs CRUD-type operations; it displays all the files or a specific file for the logged-in user and saves the files or deletes them. We use res.sendfile to display individual files because it automatically sets the correct content type and handles the streaming for us. Since the bodyParser middleware from Express was deprecated, we replaced it with connect-multiparty (a connect wrapper around the multiparty module), one of the recommended alternatives. Luckily, this module has an API similar to bodyParser, so we won't notice any differences. Check out the complete source code of files.js as follows:

var File = require('../models/file');

exports.index = function(req, res, next) {
  File.getByUserId(req.session.userId, function(err, files) {
    if (err) { return next(err); }

    res.render('files/index', {
      username: req.session.username,
      files: files,
      info: req.flash('info')[0],
      error: req.flash('error')[0]
    });
  });
};

exports.show = function(req, res, next) {
  var file = new File(req.session.userId, req.params.file);

  file.exists(function(exists) {
    if (!exists) { return res.send(404, 'Page Not Found'); }

    res.sendfile(file.path);
  });
};

exports.destroy = function(req, res, next) {
  var file = new File(req.session.userId, req.params.file);

  file.delete(function(err) {
    if (err) { return next(err); }

    req.flash('info', 'File successfully deleted!');
    res.redirect('/');
  });
};

exports.create = function(req, res, next) {
  if (!req.files.file || (req.files.file.size === 0)) {
    req.flash('error', 'No file selected!');
    return res.redirect('/');
  }

  var file = new File(req.session.userId, req.files.file.originalFilename);

  file.save(req.files.file.path, function(err) {
    if (err) { return next(err); }

    req.flash('info', 'File successfully uploaded!');
    res.redirect('/');
  });
}; 

The general routes used to require user authentication or other middleware that needs to be reused for different paths can be put inside main.js, as shown in the following code:

exports.requireUserAuth = function(req, res, next) {
  // redirect user to login page if they're not logged in
  if (!req.session.username) {
    return res.redirect('/sessions/new');
  }
  // needed in the layout for displaying the logout button
  res.locals.isLoggedIn = true;

  next();
};

The index.js file is pretty simple; it just exports all the controllers into a single object so they're easier to require in the start script of our application:

exports.main = require('./main');
exports.users = require('./users');
exports.sessions = require('./sessions');
exports.files = require('./files');

Now that we have seen what the controllers look like, we can add them to our existing app.js file:

var routes = require('./routes');
// Declaring application routes
app.get('/', routes.main.requireUserAuth, routes.files.index);
app.get('/files/:file', routes.main.requireUserAuth, routes.files.show);
app.del('/files/:file', routes.main.requireUserAuth, routes.files.destroy);
app.post('/files', multiparty(), routes.main.requireUserAuth, routes.files.create);
app.get('/users/new', routes.users.new);
app.post('/users', routes.users.create);
app.get('/sessions/new', routes.sessions.new);
app.post('/sessions', routes.sessions.create);
app.del('/sessions', routes.sessions.destroy);

Note that we included the requireUserAuth route for all the URLs that need the user to be logged in, and that the multiparty middleware is added just for the URL assigned to file uploads (which would just slow the rest of the routes with no reason).

A similarity between all the controllers is that they tend to be slim and delegate the business logic to the models.

Models

The application manages users and files, so we need to create models for both. Since the users will be saved to the database, we will work with Mongoose and create a new schema. The files will be saved to disk, so we will create a file prototype that we can reuse.

The file model

The file model is a class that takes the user ID and the filename as parameters in the constructor and sets the file path automatically. Some basic validation is performed before saving the file to ensure that it only contains letters, numbers, or the underscore character. Each file is persisted to disk in a folder named after userId (generated by Mongoose). The methods used to interact with the filesystem use the native Node.js fs module. The first part of the code is as follows:

var fs = require('fs');
var async = require('async');
var ROOT = __dirname + '/../files';
var path = require('path');

function File(userId, name) {
  this.userId = userId;
  this.name = name;
  this.path = this._getPath();
}

File.prototype._getPath = function() {
  return path.resolve(File.getUserPath(this.userId) + '/' + this.name);
};

File.prototype.isValidFileName = function() {
  return /[a-z0-9_]/i.test(this.name);
};

File.prototype.exists = function(callback) {
  if (!this.isValidFileName()) {
    // keep the function async
    return process.nextTick(function() { callback(null, false) });
  }

  fs.exists(this.path, callback);
};

File.prototype.delete = function(callback) {
  this.exists((function(exists) {
    if (!exists) { return callback(); }
    fs.unlink(this.path, callback);
  }).bind(this));
};

File.prototype.getStats = function(callback) {
  fs.stat(this.path, callback);
};

File.getUserPath = function(userId) {
  return ROOT + '/' + userId;
};

// create a folder if it doesn't exist already
File.createFolder = function(userId, callback) {
  var userPath = File.getUserPath(userId);

  fs.exists(userPath, function(exists) {
    if (!exists) {
      fs.mkdir(userPath, callback);
    }
  });
};

The most interesting methods in this model are the ones used to save a file and get all the files that belong to a user. When uploading a file, the multiparty module saves it at a temporary location, and we need to move it to the user's folder. We solve this by piping readStream into writeStream and executing the callback on the close event of the latter. The method to save a file should look like the following:

File.prototype.save = function(tempPath, callback) {
  if (!this.isValidFileName()) {
    return process.nextTick(function() {
      callback(null, new Error('Invalid filename'))
    });
  }

  var readStream = fs.createReadStream(tempPath);
  var writeStream = fs.createWriteStream(this.path);
  // if an error occurs invoke the callback with an error param
  readStream.on('error', callback);
  writeStream.on('error', callback);
  writeStream.on('close', callback);
  readStream.pipe(writeStream);
};

The function that retrieves all the files of a user reads the directory to get the files, then it calls the getStats function in parallel for every file to get its stats, and finally, it executes the callback once everything is done. In case there is an error returned because the user's folder does not exist, we call the File.createFolder() method to create it:

File.getByUserId = function(userId, callback) {
  var getFiles = function(files) {
    if (!files) { return callback(null, []); }

    // get the stats for every file
    async.map(files, function(name, done) {
      var file = new File(userId, name);
      file.getStats(function(err, stats) {
        if (err) { return done(err); }

        done(null, {
          name: name,
          stats: stats
        });
      });
    }, callback);
  };

  fs.readdir(File.getUserPath(userId), function(err, files) {
    if (err && err.code === 'ENOENT') {
      File.createFolder(userId, function(err) {
        if (err) { return callback(err); }

        getFiles(files);
      });
    } else if (!err) {
      getFiles(files);
    } else {
      return callback(err);
    }
  });
};
The User model

The only things that we need to store in the database are the users, so the user.js file contains the Mongoose schema for the User model, field validation functions, and functions related to hashing and comparing passwords (for authentication). The following code contains the module dependencies along with the validation functions and schema declaration:

var mongoose = require('mongoose');
var pass = require('pwd');

var validateUser = function(username) {
  return !!(username && /^[a-z][a-z0-9_-]{3,15}$/i.test(username));
};
var validatePassword = function(pass) {
  return !!(pass && pass.length > 5);
};

var User = new mongoose.Schema({
  username: {
    type: String,
    validate: validateUser,
    unique: true
  },
  salt: String,
  hash: String
}, {
  safe: true
});

Since we don't store the password in plain text but use a salt and a hash instead, we cannot add password as a field on the schema (in order to enforce its validation rules) nor create a virtual setter for it (because the hashing function is asynchronous). Due to this, we need to create custom functions such as setPassword, saveWithPassword, and validateAll as shown in the following code:

User.methods.setPassword = function(password, callback) {
  pass.hash(password, (function(err, salt, hash) {
    if (err) { return callback(err); }

    this.hash = hash;
    this.salt = salt;

    callback();
  }).bind(this));
};

// validate schema properties (username) && password 
User.methods.validateAll = function(props, callback) {
  this.validate((function(err) {
    if (err) { return callback(err); }

    if (!validatePassword(props.password)) {
      return callback(new Error('ValidationError: invalid password'));
    }

    return callback();
  }).bind(this));
};

User.methods.saveWithPassword = function(password, callback) {
  this.validateAll({ password: password }, (function(err) {
    if (err) { return callback(err); }

    this.setPassword(password, (function(err) {
      if (err) { return callback(err); }

      this.save(callback);
    }).bind(this));
  }).bind(this));
};

The authentication function is pretty straightforward; it gets the username and then compares the hash stored in the database with the hash generated by the password, which is sent as a parameter:

User.statics.authenticate = function(username, password, callback) {
  // no call to database for invalid username/password
  if (!validateUser(username) || !validatePassword(password)) {
    // keep this function async in all situations
    return process.nextTick(function() { callback(null, false) });
  }

  this.findOne({ username: username }, function(err, user) {
    if (err) { return callback(err); }
    // no such user in the database
    if (!user) { return callback(null, false); }

    pass.hash(password, user.salt, function(err, hash) {
      if (err) { return callback(err); }

      // if the auth was successful return the user details
      return (user.hash === hash) ? callback(null, user) : callback(null, false);
    });
  });
};

module.exports = mongoose.model('User', User);
Views

The first thing to do here is to create a global layout for our application, since we want to reuse the header and footer and only customize the unique part of every web page. We use jade as the templating language, so in order to declare the extendable part of the layout, we use the block function. The layout.jade file will be created inside the views folder as follows:

!!! 5
html
  head
    title File Store
    link(rel='stylesheet', href='http://fonts.googleapis.com/css?family=IM+Fell+Great+Primer')
    link(rel='stylesheet', href='/stylesheets/normalize.css', type='text/css')
    link(rel='stylesheet', href='/stylesheets/style.css', type='text/css')
  body
    header
      h1 File Store
      if isLoggedIn
        div
          form(action='/sessions', method='POST')
            input(type='hidden', name='_method', value='DELETE')
            input(type='submit', class='sign-out', value='Sign out')

    div.container
      block content

    script(src='http://code.jquery.com/jquery-1.10.1.min.js')
    script(src='/javascripts/file-upload.js')

Note

An interesting detail in the preceding code is that we override the method interpreted on the server side from POST to DELETE by passing a hidden field called _method. This functionality is provided by the methodOverride middleware of Express, which we included in the app.js file.

Sometimes, we need to use functions for date formatting and size formatting or as a link to use some parameters and other similar tasks. This is where view helpers come in handy. In our application, we want to display the size of the files in kilobytes, so we need to create a view helper that will convert the size of a file from bytes to kilobytes. We can replicate the same structure from the routes folder for the helpers as well, which means that we will have an index.js file that will export everything as an object. Besides this, we will only create the helper for the files at the moment, namely files.js, since that's all we need:

exports.formatSize = function(sizeInBytes) {
  return (sizeInBytes / 1024).toFixed(2) + ' kb';
};

To make the view helpers accessible inside the view, we need to add another piece of code into our app.js main file after the view setup, as shown in the following line of code:

app.locals = require('./helpers/index');

This will ensure that whatever is assigned to the locals property is globally accessible in every view file.

In the views folder, we create subfolders for files, sessions, and users. The sessions and users folders will contain a new.jade file, each with a form (user login and signup page). The biggest view file from the files subfolder is index.jade since it's the most important page of the application. The page will contain dynamic data such as the logged-in username or the number of files stored and other stuff such as an upload form and a dashboard with a list of files. The code for the index.jade file will look like the following:

extends ../layout

block content
  h2 Hello #{username}

  if !files.length
    h3 You don't have any files stored!
  else
    h3 You have #{files.length} files stored!

  if info
    p.notification.info= info

  if error
    p.notification.error= error


  div#upload-form
    form(action='/files', method='POST', enctype="multipart/form-data")
      div.browse-file
        input(type='text', id='fake-upload-box', placeholder='Upload new file!')
        input(type='file', name='file')
      button(type='submit') Go!

  if files.length
    table.file-list
      thead
        tr
          th Name
          th Size
          th Delete
      tbody
        each file in files
          tr
            td
              a(href="/files/#{encodeURIComponent(file.name)}") #{file.name}
            td #{helpers.files.formatSize(file.stats.size)}
            td
              form(action="/files/#{encodeURIComponent(file.name)}", method='POST')
                input(type='hidden', name="_method", value='DELETE')
                input(type='submit', value='delete')
Running the full application

We have not covered the JavaScript static files or stylesheets used by the application, but you can fill in the missing pieces by yourself as an exercise or just copy the example code provided with the book.

To run the application, you need to have Node and NPM installed and MongoDB up and running, and then execute the following commands in the terminal from the project root:

$ npm install .
$ npm start

The first command will install all the dependencies and the second one will start the application. You can now visit http://localhost:3000/ and see the live demo!

 

Summary


In this chapter, we learned about the main features of Express. We compared it to other existing web frameworks and discovered when it is best to use it. We saw how to structure our applications and build a practical, MVC-structured application in the process.

Coming up in the next chapter is learning about middleware in Express. We will create configurable middleware, error-handling middleware, and even our custom implementation of the middleware system, among others, so stay tuned.

About the Author
  • Alexandru Vladutu

    Alexandru Vlăduțu is a full-time JavaScript developer based in Bucharest, Romania. He started creating applications with PHP about 5 years ago, but after finding out about server-side JavaScript with Node.js, he has never had to switch technologies again. You may have seen him answering questions on Stack Overflow under the nickname alessioalex, where he is among the top three overall answerers for tags such as Node.js, Express, Mongoose, and Socket.IO. By day, he battles cross-browser compatibility issues, but by night, he brings together embedded databases, servers, and caching layers in single applications using the good parts of JavaScript. Apart from the geeky stuff, he enjoys spending time with his wife.

    Browse publications by this author
Latest Reviews (2 reviews total)
I'm just began reading it, but my first impression is very good
Mastering Web Application Development with Express
Unlock this book and the full library FREE for 7 days
Start now