Chapter 7. Developing Real-Time Web Applications

We've finally reached the last chapter of our introductory book to RethinkDB. It's been a long journey so far!

In the previous chapters, you learned how to administer RethinkDB and write and run queries using the ReQL language. You also learned how to administer a cluster, run queries from Node.js, and deploy RethinkDB to the cloud.

In this chapter, we're going to put everything that we've covered together; we'll understand what real-time apps are, how they work, and how to use RethinkDB's features to power a Node.js web application.

In this chapter, you will also learn about the following:

• How to install and use the Express.js module

• How to connect a Node.js backend to a .js frontend

• How to use Changefeeds to push data to the client

At the end of the chapter, you'll have a perfectly working Node.js web application that you can use as a template when developing your own web apps. We'll also have a brief recap of the most important things that we...

Introducing real-time web applications

In this chapter, we're going to see how we can use the tools we've covered so far to build real-time applications for the Web. However, before diving into it, one of the first questions you may want answered is, "what exactly is a real-time web application?"

A real-time web application can be defined as any kind of web-facing software that allows clients (usually users) to receive information as soon as it gets published. In other words, it's a software application that functions within a timeframe that the user senses as immediate or current. In these types of software, latency must be extremely low, usually in the range of milliseconds.

Examples of real-time web apps

You may be wondering what a real-time web app looks like. Well, chances are that you use one of them every day! Gmail is an example of a real-time web app; we've all noticed e-mails and notifications popping up on our browser in real time and an asynchronous manner without the need of refreshing...

Going real time on the Web

Real-time web applications have traditionally been quite complex to develop as they once relied on a series of hacks and workarounds. This is the reason why many developers avoid going real time as they believe it will bring in more complexity to their software backends. To fully appreciate and understand this concept, it's important to know how real-time applications are implemented and how they've evolved over time.

As you may know, web applications are built over the HTTP protocol; however, for quite some time, HTTP and real-time applications didn't really go together very well. The reason for this is that the HTTP protocol wasn't built for this purpose, but it was built with client-to-server communications in mind.

Let's clarify this a bit more. At the heart of this protocol, there is a stateless request-response cycle—the client executes a request and an HTTP server reacts upon these requests, providing a response back to the client, but it doesn't keep a persistent...

Developing web applications with Node.js

In the previous chapter, you learned all about Node's key features that make it a great choice when writing highly-performing software and when you need to interact with a database. You'll be happy to know that Node.js is also a great choice when you want to build real-time web applications.

Node.js allows you to build modern, real-time web applications that use a module called Socket.io that implements WebSockets, allowing you to push data asynchronously from the server to the clients. This technology, combined with RethinkDB's Changefeeds, allows you to build web applications that send data to the clients in real time without all the complexity and downfalls of traditional real-time programming flows, such as continuously polling the server for updates.

When developing web applications, most languages and platforms provide you with frameworks that greatly simplify the job of writing TCP or HTTP servers. For example, the most popular framework for...

Using RethinkDB in Node.js web applications

When you develop web applications of any kind, more than often, you will need a database to store data in. Without a proper persistence layer, only basic web apps can be built. In Chapter 5, Programming RethinkDB in Node.js, you learned how to connect to a RethinkDB cluster and run queries from a Node.js script. In this chapter, we're going to use the same syntax to use RethinkDB within an Express.js web app.

Fortunately, adding database connectivity capability to Express.js web apps is just a matter of loading an appropriate Node.js driver for the database in your app; however, you maybe wondering where exactly we put the code that interacts with the database, in the context of our Express.js web application. The answer lies within the MVC pattern.

If you've ever developed web applications before, you'll certainly be familiar with the model-view-controller pattern, which divides an application into three separate components: the model that stores...

Your first real-time web application

To get a better understanding of how all these components fit together, we're going to develop our first real-time web application based on Node.js and RethinkDB. If you've never developed a web app before, this is going to be very exciting for you!

You maybe wondering what we're going to build. We're going to develop a web application that allows us to create colored "notes" containing messages that can be dragged around the screen. The interesting thing about this web application is that it's completely real time. Everything you do will be reflected in real time on all connected clients. What this means is that if you open two browser windows and drag a note around, the same note will be animated in the other browser window. The same thing happens when you create or delete notes. All the changes will appear in real time in all the open browser windows without the need of reloading the page.

If you're curious, the application we're going to build looks...

Creating the Node.js server

The first step in developing the notes web application is coding the HTTP server that will serve the clients' requests, so we're going to start with the app.js file.

The HTTP server relies on the Express.js module, so we first need to import the module and all other modules that will be used in the app.

Take a look at the following code:

var express = require('express'),
app = express(),
server = require('http').createServer(app),
io = require('socket.io')(server),
path = require('path'),
bodyParser = require('body-parser'),
model = require('./models/rethinkdb');

These seven lines actually do quite a bit of work! Let's go through them line by line.

The first line imports the Express.js module and makes it accessible through the express variable so that we can use it in the script.

Line number two creates a new instance of an Express.js app, while line three imports the http module and uses the app variable to create an instance of an HTTP server.

Line number four imports...

Configuring the router

When you design the routes for a web application, you should think about all the features that the application should provide to the user and map these features to an HTTP request.

In our example app, the user will need to load the homepage, get a list of all existing notes, create a new note, update a note's position, and delete a note. What we should do now is create an API for our app, which creates a route for all of these actions. I've personally organized it with the following routes:

• get (/: This route gets called when the user visits the app's homepage from his browser and is responsible for rendering the view to the screen

• get (/notes: This route allows us to request a list of all existing notes that are stored in the database

• post (/notes: This route is called when the user creates a new note

• post (/note/:id: When the user moves a note, a request to this route is performed, passing the note's ID

• delete (/note/id: Finally, this route allows us to delete a specific...

Writing the application logic

Probably, the most complex aspect of developing a web application is the controller as this is where all the logic is implemented.

In our example app, the controller will be responsible for connecting the model to the view; in other words, the controller will call a function from the model, receive some data, structure it in the correct format, and send it to the browser.

In the previous section, we structured the router dividing each feature of the app and calling a different function for each feature. In the controller, we need to implement these functions. As an example, let's write the controller function that updates the position of an existing note. In the router, we defined this function as updateNote(), so we can implement it as follows:

controller.updateNote = function (req, res, model) {
    var note = new Object();
    note.id = req.params.id;
    note.xpos = req.body.xpos;
    note.ypos = req.body.ypos;
    model.updateNote(note, function (success,...

Interacting with RethinkDB

Now that we've written both the router and the controller, it's time to turn back to RethinkDB and write the code that physically saves and loads data from the database. In the previous section, you learned how different functions in the controller call different functions in the model, so what we need to do now is implement these functions in the models/rethinkdb.js file.

As an example, let's focus on the function that creates a new note and saves it to the database: the user first creates the note from the app in his browser, then the browser performs an HTTP request that is received from the Node.js server and passed on to the router. At this point, the router handles the request and passes the data to the controller, which, in turn, extracts the data and calls the appropriate function in the model. In this case, the function to save a note in the model is called saveNote() and is implemented as this:

model.saveNote = function (note, done) {
    r.connect({ host...

Programming Socket.io

Probably, the top feature in our web application is its ability to send updates directly from the server to the client in real time. To fully implement this feature, we need to call the model's listenForUpdates() function and push the resulting data to all connected clients.

At the start of this chapter, you learned about the Socket.io module, which allows you to send data to clients using WebSockets; so, we're going to use this module to push changes to our users' browsers. You may remember that we initialized the Socket.io module in the app.js file, so we're now going to edit the file; in particular, we're going to edit the server.listen() function:

server.listen(8000, function() {
    console.log"Server up and listening on port 8000");
    model.listenForUpdates(function(data) {
        if((data.new_val != null) && (data.old_val != null)) {
            // position update
            io.emit("updates", data.new_val);
        } else if((data.new_val != null)...

Programming the frontend

When the user visits our web application's homepage, the index.ejs template is rendered to screen. This template is actually just a simple HTML document that links to a client-side JavaScript file called notes.js.

If we take a look at the EJS template, we can see exactly where the JavaScript file is loaded:

<script src="notes.js"></script>

You maybe wondering where this file comes from. Actually, this JavaScript file is the one we saved into the public folder of our web application. This script is executed from the user's browser as soon as the HTML page has finished loading.

This script is responsible of doing a few things:

• Adding drag and drop capabilities to the notes

• Initializing Socket.io

• Listening for updates

• Updating the graphics to reflect changes

  Note

  As the focus of this book is interacting with RethinkDB, we won't go over the code that adds drag-and-drop capabilities to the notes or the code that works with the graphics. Instead, we'll focus on how to...

The view

The final component that completes our web application is, of course, the view. Our app has only one view, the homepage. This view is described as an EJS template in the index.ejs file that you can find in the views folder. Let's take a look at this file to understand its structure:

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>Real-time Notes</title>
<link  href="http://fonts.googleapis.com/css?family=Reenie+Beanie:regular" rel="stylesheet" type="text/css">
<link rel="stylesheet" href="notes.css">
<script src="http://code.jquery.com/jquery-1.11.0.min.js"></script>
<script src="http://code.jquery.com/ui/1.10.4/jquery-ui.js"></script>
<script src="http://cdnjs.cloudflare.com/ajax/libs/magnific-popup.js/0.9.9/jquery.magnific-popup.min.js"></script>
<script src="/socket.io/socket.io.js"></script>
<script src="notes.js"></script>
<link href="http://cdnjs.cloudflare...

Running our web application

Now that we've written all the code necessary to start our web app, we can try running it. You may remember that the app's entry-point is the app.js file.

We can start our web application by running the following command:

node app.js

If everything works correctly, you will receive no errors and get a few logging messages similar to this:

[127.0.0.1 2015-11-23T09:08:14.739Z] Setting up RethinkDB...
[127.0.0.1 2015-11-23T09:08:14.749Z] Server up and listening on port 8000

As you can see, our Node.js server is running on port 8000, so you access the web app by opening your browser and visiting http://YOUR_IP_ADDRESS:8000.

When the page loads, our web application will look as this:

You can try creating new notes by clicking on the +button and writing a message. If you open the web application in two different windows in your browser, you will see how everything you do in one window is reflected in the other. Try dragging around the note in one window, and you will see...

Summary

We've come to the end of this rather dense chapter and this ends our book. This chapter has been full of new concepts, and I hope it helped you understand RethinkDB's true potential as a document database.

In this chapter, you learned all about what a real-time web app is and how it works. We then proceeded to analyze each of the components that together constitute the structure of a Node.js web application and how RethinkDB interacts with them. Finally, we developed our first full-fledged, real-time app and tested it in the browser.

This concludes our journey! You learned about a lot of different topics, and you should now have the knowledge required to build your own apps using RethinkDB—the database for the real-time web.