Building Bots with Node.js

In the broad sense of its definition, a bot is a piece of software that leverages artificial narrow intelligence to perform specific tasks in place of a human. Bots understand language to a certain extent and not just commands. Ultimately, they could learn from their interactions to get smarter and better.

In roughly two years time, 3.6 billion people (yes 3.6 billion) are projected to be using messaging apps--that's 90% of total Internet users, which is more people than could ever be served with a continuous thread of communication compared to more traditional platforms such as e-mail. Refer to the following link for more information:  https://hbr.org/2016/09/messaging-apps-are-changing-how-companies-talk-with-customers .

Worldwide, consumers are now demanding messaging as a customer service option. It's not sufficient to have a customer service phone number where the customer can call you, but it's becoming almost a must that customers should be able to reach you through some kind of real-time messaging platform as well. Users are demanding fast-paced interaction and quick answers.

Recent studies found that messaging and chat were the highest rated contact methods for customer satisfaction. Refer to  https://onereach.com/blog/45-texting-statistics-that-prove-businesses-need-to-start-taking-sms-seriously/ .

According to recent polls ( http://customerthink.com/7-data-backed-reasons-why-you-should-let-customers-text-customer-service/ ), almost two-thirds of consumers are likely to have a positive perception of an organization that offers messaging or chat as a service channel. Nevertheless, by the end of 2016, roughly 40% of customer service centers will still be missing that opportunity to impress their customers. This translates not only into failing to impress your customers, but also as a loss of business opportunities. Customers are likely to be more loyal and stay with those organizations that are capable of interacting and engaging with them in faster and smarter ways. Refer to  https://blog.kissmetrics.com/live-chat/ .

Consider your organization (corporate) has a messaging app that allows your customers to interact with you. Even though your app might be a great communication gateway, there's still no room for that communication channel to be lost. Say, for instance, a user forgets to turn on notifications or accidentally deletes the app. The ability to seamlessly and easily communicate is suddenly gone.

However, using a personal messaging app (such as Skype, Facebook Messenger, WhatsApp, and so on) eliminates most roadblocks, allowing for companies to become part of the communication framework that users already know and love.

With messaging apps, there are no forms, no downloads, no new platforms. The customer can use the interface that they are already familiar with to instantly engage with your organization. The user can use natural language to purchase a ticket, download a boarding pass, or ask a question. Moreover, given that the user is highly unlikely to stop using the messaging app, your organization can follow up with updates, surveys, and other notifications through the messaging app that the user already knows and loves.

In order to understand this better, say when a consumer asks a question, the bot should be able to:

There will surely be cases where bots might encounter situations that require the nuance and analytical thinking of a human. When they do, they can escalate to an agent, passing along the context they've gathered during the interaction to ensure a seamless customer experience. In principle, this should be totally transparent for the end user.

As technology continues to advance, Gartner predicts that by 2018, bots should be able to recognize customers by face and voice rather seamlessly.

Bots could also be able to:

Smart phones are becoming more important in today's world. Arguably, they are almost an extension of yourself. If you lose your phone today, you are in trouble. Everything from e-mails, calendar, messaging, banking, and even your wallet are somehow linked to your phone.

In today's vibrant, dynamic, and always connected society, having access to vast amounts of information at your fingertips through your phone can be a blessing, but it can also be a curse.

Busy professionals nowadays have to deal with hundreds of e-mails on a daily or weekly basis, plus also many messages and notifications from social networks such as Twitter and LinkedIn. Keeping up with this sheer volume of messages can be overwhelming.

But what if phones could actually help us alleviate some of this information overload by notifying us of important things or allowing us to perform custom actions based on SMS or voice commands? Imagine if we were able to automate certain processes through messaging or voice. Wouldn't that be awesome?

Before social networks took off, Short Message Service (SMS) was the most common way to exchange short messages between people.

According to Wikipedia, even though SMS is still strong and growing, social networking messaging services such as Facebook Messenger, WhatsApp, Skype, and Viber, available on smart phones, are increasingly being used to exchange short messages.

Generally speaking, SMS and voice enabled solutions are platform specific and cannot be customized; however, there's a platform that was designed from the ground up with developers in mind, which allows anyone with development skills to create custom messaging and voice enabled solutions. Welcome to Twilio! refer to  https://www.twilio.com/ .

Twilio is a messaging, voice, video, and authentication API for every application. It has helper libraries or SDKs in many different programming languages that help developers create apps that can leverage the power of voice and messaging.

Despite that, SMS is still very strong and widely used in enterprise development for things such as marketing, Customer Relationship Management (CRM) automation, real-time alert notifications, and two-step verification of a user's identify.

The significance of SMS usage in the business world is incredibly important given that the technology is considered mature, widely spread, proven, and reliable.

Twilio's services are accessed over HTTP(S) through a RESTful API or helper libraries. Its services are billed based on usage. The platform is based on Amazon Web Services (AWS) to host its telephony infrastructure and provide connectivity between HTTP and the Public Switched Telephone Network (PSTN), through its APIs.

Twilio has recently extended its API support to Facebook Messenger, which coincides with the social networking company's introduction of support for bots on its Messenger platform.

In this chapter, we'll explore how to interact with Twilio's REST API using the Node.js helper library in order to build an SMS Messaging bot.

Installing Twilio for Node.js

Twilio provides a REST API, which allows developers to interact with its platform services, such as SMS. Even though the REST API is a great way to interact with Twilio services, there are official helper libraries for the most common programming languages of today, such as: PHP, ASP.NET (C#), Ruby, Python, Java, Salesforce (Apex), and, last but not least, Node.js.

The Twilio Node.js helper library can be obtained from https://www.twilio.com/docs/libraries/node . In order to get started, let's get Node.js installed.

Open your browser and navigate to https://nodejs.org and there on the main page you can download the version of Node.js that corresponds to your platform.

The steps that follow will be based on installing Node.js on a Windows 64 Bit operating system.

Once you have selected a version, just run the installer and follow the installation steps. You'll first be presented with a Welcome screen and then you can click on the Next button.

You'll be requested to accept the license terms and then click on the Next button again.

Following that, the installer displays the default installation path, which you may opt to change or not.

Once the installation path has been defined, simply click on the Next button. The next step is to select what features will be installed.

It is highly recommended to leave all the features selected so everything can be installed. The npm package manager will be later required in order to install the Twilio Node.js helper library.

Finally, click on the Next button and then the Install button, in order to finalize the installation process.

If there was a previous version of Node.js installed on your system, the installer will remove previous older files and then update the system with the newest files.

Please note that on other platforms ( https://nodejs.org/en/download/package-manager/ ), the installation process and screens might differ (such as on a Mac); however, it should be pretty straightforward and easy to follow along by going through the installation steps.

Once Node.js has been installed, the next thing to do is to get the Twilio Node.js helper library installed.

In order to do this, create a folder anywhere on your PC for this project, browse to this folder, and then open the Command Prompt or shell and type this command:

npm init

Just follow the steps requested. This will create the package.json file (you can refer to https://docs.npmjs.com/files/package.json ) required for our project.

Once the package.json file has been created, type in the following command:

npm install twilio --save

This will install the Twilio Node.js helper library and all its dependencies and save the reference on our package.json file. The Twilio library will be installed under the node_modules folder within the folder where your package.json file resides. We'll be using the awesome Atom editor ( https://atom.io/ ) throughout this book. You may use any other editor of your choice, such as Sublime or Visual Studio Code.

With this in place, we can technically start writing code. However, we first need to sign up for a Twilio account and get all set up with Twilio before we can send our first SMS. Let's explore how we can get this done.

Setting up a Twilio account

In order to be able to send SMS using the Twilio API and Node.js helper library, we need to get a Twilio account set up and also purchase a disposable Twilio number.

Twilio is a pay-as-you-go service, which means that you'll need to set up an account and provide your credit card details in order to have enough credit, which will be used to pay for every SMS you send.

You'll also need to purchase a Twilio number, which is a regular but disposable phone number that will be used to send your messages.

Twilio numbers are available for many countries. They look like any other valid phone number you can think of. They are real phone numbers that you can dispose of when you no longer need them.

In order to set up a Twilio account, from your browser access the following site https://www.twilio.com/ . Then, click on the SIGN UP button.

The sign-up process is fairly straightforward, and is super easy to follow and complete. Just fill in a few fields that are required and you're done.

Once your Twilio account has funds, you'll need to purchase a disposable phone number.

You'll need to go to this location,  https://www.twilio.com/user/billing , in order to add funds to your account. In order to do that, click on the Ad Funds link in red. Make sure that you have logged into Twilio before accessing this URL.

With funds in your account, let's set up a Twilio number. This will be a real phone number, which you can delete at any moment. You may choose from which country and city your number will belong to.

Then click on the Buy a Number button.

Once you have clicked on the Buy a Number button, the following pop-up screen will be shown:

On this screen, you have the option to choose which country you would like to get the number from and also from which geographical location.

The number could be used for Voice, SMS, and even MMS. For now, we are simply interested in making sure that the SMS option is ticked.

Once you have purchased your Twilio number, you will see the following screen:

With this in place, we are ready to start writing our Node.js code.

Bare-bones Twilio Node.js template

In order to start writing our code, let's create a new file in the same location as our package.json file called app.js. You may create this new file directly from the editor you are using.

Once the file has been created, we'll need to include a reference to the Twilio Node.js library that we installed through npm:

var twilio = require("twilio"); 

In the Node.js world, this is the equivalent of an import in Java or using statements in C#. Now let's move on to see how we can actually send an SMS using the Twilio Node.js helper library:

var accountSid = '<< your twilio account sid >>';  
// Your Account SID from www.twilio.com/console 
 
var authToken = '<< your twilio auth token >>';    
// Your Auth Token from www.twilio.com/console 

We'll need two variables to store our Twilio Account SID and our Auth token. Both values can be obtained when you log in to your Twilio account and browse to the developer console: https://www.twilio.com/console .

Once we have provided the correct values to the accountSid and authToken variables, we'll need to create an instance of the twilio.RestClient class in order to be able to send an SMS:

var client = new twilio.RestClient(accountSid, authToken); 

With our instance created, we can go ahead and send our SMS using Twilio:

client.messages.create({ 
    body: 'Greetings earthling, this is the TwilioSmsBot ;)', 
    to: '+12345678901',  // Number that receives the SMS 
    from: '+12345678901' // Purchased Twilio number that send the SMS 
}, 
function(err, message) { 
    console.log(message.sid); 
}); 

Basically, the SMS is sent by invoking the messages.create method from the Twilio client instance.

This method expects an object that describes the properties of the SMS, such as the body, to (receiver number), from (sender number), call back function which describes an error err (if an error actually happens), and the contents of the posted message.

This is all that is required in order to send an SMS using Twilio. Let's have a look at all the code now:

var twilio = require("node_modules/twilio/lib"); 
 
var accountSid = '<< your twilio account sid >>';  
var authToken = '<< your twilio auth token >>';    
 
var client = new twilio.RestClient(accountSid, authToken); 
 
client.messages.create({ 
    body: 'Hello from Node', 
    to: '+12345678901',       
    from: '+12345678901' 
}, 
function(err, message) { 
    console.log(message.sid); 
}); 

In order to execute this code, execute this command from the Command Prompt:

node app.js

This will send the SMS to the number indicated. We can see the message.sid of the SMS sent (which was sent back as a response from the Twilio service) by looking at the Command Prompt.

Please notice that if the destination number you will be messaging is an international (non-US) number, you'll need to enable certain permissions to allow Twilio to perform that action.

These permissions can be checked and configured at this URL: https://www.twilio.com/console/voice/settings/geo-permissions .

Now that we've implemented and have a working Twilio Node.js template, which can send SMS, let's have a look at expanding our code to do more.

We'll need to be able to somehow hook and listen to incoming SMS and have some very basic Natural Language Processing (NLP), in order to send answers based on the input received.

Listening to incoming SMS requires setting up in our Node.js app a URL that can be configured within your Twilio account as a Request URL. This Request URL will be used by Twilio to push incoming messages on your purchased Twilio number, to our Node.js bot app.

To make our bot publicly available, we'll publish it on Azure websites. Let's create a REST endpoint for our Node.js app, which we will use for listening to new messages. We'll be using the Express framework (http://expressjs.com/) to do this.

Express is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications. It provides a thin layer of fundamental web application features such as routing and middleware.

Let's first install the Express framework by running this command from the command line prompt:

npm install express --save

A handy utility to have installed is Nodemon (http://nodemon.io/). This allows you to make changes to your code and it automatically restarts the Node.js app. You can install Nodemon by running this command from the prompt:

npm install nodemon --save

Instead of running the app with the node, you can now run it as follows:

nodemon app.js

With the Express framework installed, let's expand our current code to create a REST endpoint, which we can then use to hook up Twilio to push incoming messages:

var express = require('express'); 
 
var app = express(); 
 
app.get('/receive', function (req, res) { 
  res.send('Hi, this is the TwilioBot listening endpoint!'); 
}); 
 
app.listen(8080, function () { 
  console.log('TwilioBot listening on port 8080.'); 
}); 

Before we can hook up our bot app to Twilio in order to process incoming messages, let's first get all the tooling wired up, so that we can publish our app as it is to Azure websites.

We'll need to install the Azure Command Line Interface (CLI) (https://azure.microsoft.com/en-us/documentation/articles/xplat-cli-install/) in order to push our app to Azure. We'll also need to sign up for Azure if we don't have an account.

You can do that by visiting: https://azure.microsoft.com .

Once you have your account set up with Azure, you may install the Azure CLI using the respective installer for your platform or as an npm package following these instructions.

Using npm, the Azure CLI can be installed as follows:

npm install azure-cli -g

Once you have installed the Azure CLI, let's deploy the app as it is to Azure in order to make sure all our tooling is correctly wired up.

In order to do that, run the Azure CLI and login to Azure:

azure login

Once the command has been executed, you'll see the following welcome message where you'll be asked to enable data collection:

You may opt-in or not, the choice is yours and this doesn't affect your bot app development or usage of Azure at all.

Once you've chosen your option, you'll be prompted to enter the code displayed on the command line on this URL, http://aka.ms/devicelogin , and then authenticate with your Microsoft Account, as follows:

In the preceding screenshot, I've blanked out the Azure subscription keys and information that corresponds to my Azure account.

With this in place, your Azure CLI is all set. The next thing to do is to deploy the app to Azure using the CLI. Let's see how this can be done.

Run this command in order to create the website on Azure. Make sure you're still in the root directory of your app. Create the App Service app resource in Azure with a unique app name with the next command. Your web app's URL will be http://<appname>.azurewebsites.net.

In this case, we'll call our App Service app on Azure: NodeBotSite (you are free to choose any other name if this has been taken). Let's enter the following command:

azure site create --git nodebotsite

You'll be prompted to select the Azure region where your site will be hosted on. Feel free to choose the one closest to your location.

Once you've selected the region, Azure will create your site and you'll see the following details via the command line:

Change the port from 8080 within the app to process.env.port, as follows:

app.listen(process.env.port, function () { 
  console.log('Hi, this is the TwilioBot listening endpoint!'); 
}); 

Save your changes on both your package.json and app.js files, and then use the git command to deploy your app to Azure, as follows:

git add .
git commit -m "TwilioNodeBot first commit"
git push azure master

After you type these commands, if you've never set up git/FTP deployment credentials for your Azure subscription, you'll also be prompted to create them. You can also enter these credentials on the Azure Portal.

Once the git push command has finished, your app will be published on Azure and it is ready to be used.

In order to view it, open your browser and navigate to the site, receive this URL: http://nodebotsite.azurewebsites.net/receive . You should then see this:

To make updates to your Node.js web app running on Azure, just run git add, git commit, and git push like you did when you deployed it first.

With Azure all wired up, the next step is to configure this URL: https://www.twilio.com/console/phone-numbers/incoming , on your voice number's dashboard within your Twilio account, and then click on your purchased Twilio number. Edit the Messaging | Request URL for your Twilio number and click on the Save button.

Now your Twilio is tied to your Azure app, receive URL, which will be used to receive incoming SMS.

With all these setup steps in place, we can now focus on adding the receive logic for our Twilio bot app.

So far we've implemented the basic bare-bones template for our Twilio bot app and also made all the necessary configurations in order to have our solution wired up with Twilio and also easily deployable to Azure.

Let's now explore how we can make our bot reply to incoming messages. In order to do this, we'll need to have a POST endpoint on our Node/Express app. Let's examine the following code:

app.post('/receive', function (req, res) { 
  var twiml = new twilio.TwimlResponse(); 
  twiml.message('Hi, this is TwilioBot'); 
 
  res.writeHead(200, {'Content-Type': 'text/xml'}); 
  res.end(twiml.toString()); 
}); 

We can see here that in order to reply, we create a TwiML response and send that as the response of POST / receive the HTTP endpoint.

TwiML is an XML markup language, which is simply a set of instructions you can use to tell Twilio what to do when you receive an incoming call or SMS.

Twilio makes HTTP requests to your application just like a regular web browser. By including parameters and values in its requests, Twilio sends data to your application that you can act upon before responding. This is what we are actually doing on this, receive endpoint.

Twilio sends the following parameters with its request as POST parameters or URL query parameters, depending on which HTTP method you've configured.

When we receive an SMS or a phone call on our Twilio number, Twilio will fetch the URL associated with that phone number and perform an HTTP request to that URL. This URL will contain an XML response with markup instructions, which indicate what tasks Twilio needs to execute. Some of these tasks can be to record the call, play a message, prompt the caller to enter some digits, and so on.

In this case, what our bot is doing is simply returning a one-line sentence. In short, the preceding example code is simply returning this XML back to Twilio as a response, so that Twilio can actually generate an SMS response out of it, and send it back to the sender's phone:

<?xml version="1.0" encoding="UTF-8"?> 
<Response> 
<Say> Hi, this is TwilioBot.</Say> 
</Response> 

As you can see, it is super simple to tell Twilio to execute a specific action using TwiML. The markup language is made up of verbs highlighted in blue, which represent actions that Twilio will execute.

Some of the TwiML verbs available at the time of the writing are: Say, Play, Dial, Record, Gather, Sms, Hangup, Queue, Redirect, Pause, Conference, Reject, and Message.

Complete details on how to use these TwiML verbs can be found here: https://www.twilio.com/docs/api/twiml .

What we've done through the code is to use the Twilio Node.js helper library to generate the TwiML as a response, without explicitly creating the XML response itself.

So our bot starts to finally take shape. It can listen to messages and send a response back. But, how does the bot know how to act to certain inputs? In order to achieve that, the bots need to be able to understand the parameters of the incoming message and be able to act upon it.

Twilio sends several parameters with its request as POST parameters or URL query parameters, depending on which HTTP method you've configured on the Twilio number's dashboard for incoming SMS. Please take a moment to go through and understand this list thoroughly.

Here are some of the prominent properties that Twilio uses:

The full list of parameters can be found here: https://www.twilio.com/docs/api/twiml/sms/twilio_request .

So in order to write some logic that acts upon the input of the nature of the messages received, it is necessary to inspect the value of the parameters that are received as part of the request.body object. These parameters will have properties on this object.

So for instance, if we want to know from which number the message came from, we would have to do something like this:

var from = req.body.From; 

The actual received text message itself would be obtained as follows:

var body = req.body.Body; 

Knowing from where the message originates and the actual contents, we can then build some logic to internally give the bot some sense of what to do with the input received and act upon it.

So, let's add some core functionality, which for now we will call BotBrain and we don't really know what it will do. This will give us an answer based on the input we provide to it, independently of which bot we will be building.

In the next chapter, we'll be creating the logic of this BotBrain, and throughout the chapters keep adding subsequent and additional functionality to it, but for now, let's assume this BotBrain gives the bot the answer required based on the input received:

app.post('/receive', function (req, res) { 
  var twiml = new twilio.TwimlResponse(); 
  var feedback = BotBrains(req.body); 
  twiml.message(feedback); 
 
  res.writeHead(200, {'Content-Type': 'text/xml'}); 
  res.end(twiml.toString()); 
}); 

As we can see now, the bot will respond to the incoming request based on the results the BotBrains will determine, by analyzing the properties of the request.body object.

In this chapter, we've set the foundations on how our bot will be deployed, configured, and set up. We've also created a bare-bones template that we will use throughout the rest of the chapters in this book, in order to add more exciting functionalities.

We've also explored some of the key components that will make it work from an infrastructure point of view and how we can host it on Azure, Microsoft's awesome cloud platform.

Beyond this, we've explained why bots matter and why they are a key component for your business to be aware of and considered in your strategy.

Further to that, we've explored the basics of Twilio as a messaging platform and started to scratch the surface of what is possible to achieve with it.

In the chapters to follow, we'll be adding many more layers and interacting with other APIs and services; however, we'll still use Twilio as a backup messaging provider by also using SMS.

Hopefully you've got an idea of what to expect coming ahead and also this has gotten you excited in our quest to add more layers and logic to our bot.

Thanks for reading!