Building Bots with Microsoft Bot Framework

4 (3 reviews total)
By Kishore Gaddam
    Advance your knowledge in tech with a Packt subscription

  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Setting up Microsoft Bot Framework Dev Environment

About this book

Bots help users to use the language as a UI and interact with the applications from any platform. This book teaches you how to develop real-world bots using Microsoft Bot Framework.

The book starts with setting up the Microsoft Bot Framework development environment and emulator, and moves on to building the first bot using Connector and Builder SDK. Explore how to register, connect, test, and publish your bot to the Slack, Skype, and Facebook Messenger platforms.

Throughout this book, you will build different types of bots from simple to complex, such as a weather bot, a natural speech and intent processing bot, an Interactive Voice Response (IVR) bot for a bank, a facial expression recognition bot, and more from scratch.

These bots were designed and developed to teach you concepts such as text detection, implementing LUIS dialogs, Cortana Intelligence Services, third-party authentication, Rich Text format, Bot State Service, and microServices so you can practice working with the standard development tools such as Visual Studio, Bot Emulator, and Azure.

Publication date:
May 2017
Publisher
Packt
Pages
424
ISBN
9781786463104

 

Chapter 1. Setting up Microsoft Bot Framework Dev Environment

In the past several decades, the corporate, government, and business world has experienced several waves of IT architecture foundations, moving from mainframes, to minicomputers, to distributed PCs, to the Internet, to social media / mobile, and now to the Cloud Internet of Things (IoT) stack. We call this the sixth wave of corporate IT, and like its predecessors, cloud and IoT technologies are causing significant disruption and displacement, even while they drive new levels of productivity. Each architecture focuses on key business processes and supports killer technology applications to drive new levels of value. Very soon we will be looking at an enormous networked interconnection of everyday machines to one another, as well as to humans.

Lets have a look at the fifth wave of corporate IT:

The machine-to-machine-to-human connectivity will have a profound impact on the consumer and corporate IT experience. As these machines become social and talk to us, we have an enormous opportunity to greatly enhance their value proposition through improved product quality, customer experience, and lowered cost of operations. A heightened consumer expectation for more personal and real-time interactions is driving business to holistically embrace the next wave of technology innovation such as cloud, IoT, and bots to boost business performance. In this age of billions of connected devices, there is a need for such a technology where our apps, such as bots, could talk back. Bots that have specific purposes and talk to any device or any app or to anyone, live in the cloud, we can talk to via any communication channel such as e-mail, text, voice, chat, and many others, can go where no apps have gone before when it comes to the machine-to-machine-to-human connectivity. In order to make this happen, we will need a whole new platform, a platform for conversations.

 

Conversation as a Service (CaaS)


Messaging apps in general are becoming a second home screen for many people, acting as their entry point to the Internet; where the "youngins" are, the brands will follow. Companies are coming up with messaging apps as bots and apps that offer everything from customer service to online shopping and banking.

Conversations are shaping up to be the next major human-computer interface. Thanks to advances in natural language processing and machine learning, the technology is finally getting faster and accurate enough to be viable. Imagine a platform where language is the new UI layer. When we talk about conversation as a platform, there are three parts:

  • There are people talking to people. The Skype translator is an example where people can communicate across languages.
  • Then, there is the opportunity to enhance a conversation by the ability to be present and interact remotely.
  • Then, there are personal assistants and the bots.

The following screenshot shows the Conversation as a Service:

Think of bots as the new mechanism that you can converse with. Instead of looking through multiple mobile apps or pages of websites, you can call on any application as a bot within the conversational canvas. Bots are the new apps, and digital assistants are the meta-apps. This way, intelligence is infused into all our interactions.

 

This leads us to the Microsoft Bot Framework, which is a comprehensive offering from Microsoft to build and deploy high quality bots for your users to interact using Conversation as a Platform (CaaP). This is a framework that lets you build and connect intelligent bots. The idea is that they interact naturally wherever your users are talking, such as Skype, Slack, Facebook Messenger, text/SMS, and others. Basically, with any kind of channel that you use today as a human being to talk to other people, you will be able to use them to talk to bots, all using natural language:

The Microsoft Bot Framework is a Microsoft operated CaaP service and an open source SDK. The Microsoft Bot Framework is one of the many tools that Microsoft is offering for building a complete bot. Other tools include Language Understanding Intelligent Service (LUIS), Speech APIs, Microsoft Azure, Cortana Intelligence Suit, and many more.

 

Your bot


The Microsoft Bot Builder SDK is one of three main components of the Microsoft Bot Framework. First, you have to build your bot. Your bot lives in the cloud and you host it yourself. You write it just like a web service component using Node.js or C#, like the ASP.NET Web API component. The Microsoft Bot Builder SDK is open source, so it will support more languages and web stacks over time. Your bot will have its own logic, but you also need a conversation logic using dialogs to model a conversation. The Bot Builder SDK gives you facilities for this, and there are many types of dialogs that are included, from simple yes or no questions, to full LUIS, which is one of the APIs provided by Microsoft Cognitive Services. This is what the architecture of bot looks like:

 

The Bot Connector


The Bot Connector is hosted and operated by Microsoft. Think of it as a central router between your bots and many channels to communicate with your bots. Apart from routing messages, it manages state within the conversation. The Bot Connector is an easy way to create a single backend and then publish it to a bunch of different platforms called channels.

The following screenshot illustrates the Bot Connector:

 

The Bot Directory


The Bot Directory is where the user will be able to find bots. It's like an App Store for mobile apps. The Bot Directory is a public directory of all the reviewed bots registered through the developer portal. Users will be able to discover, try, and add bots to their favorite conversation experiences from the Bot Directory. Anyone can access it and can submit bots to the directory.

As you begin your development with the Microsoft Bot Framework, you might be wondering how best to get started. Bots can be built in C#; however, Microsoft's Bot Framework can also be used to build bots using Node.js. For developing any bots, we need to first set up the development environment and have the right tools installed for successfully developing and deploying a bot. Let's see how we can set up a development environment using Visual Studio.

 

Setting up the development environment


In this section, we will see how to set up the development environment but, before that, let's check out the prerequisites needed for setting it up.

Prerequisites

To use the Microsoft Bot Framework Connector, you must have the following:

  • A Microsoft account (Hotmail, Live, or Outlook) to log into the Bot Framework developer portal, which you will use to register your bot.
  • An Azure subscription (free trial at https://azure.microsoft.com/en-us/). This Azure subscription is essential for having an Azure-accessible REST endpoint exposing a callback for the Connector service.
  • Developer accounts on one or more communication service (such as Skype, Slack, or Facebook) where your bot will communicate.

In addition, you may wish to have an Azure Application Insights account so that you can capture telemetry from your bot. There are additionally different ways to go about building a bot: from scratch, coded directly to the Bot Connector REST API, the Bot Builder SDK's for Node.js and .NET, and the Bot Connector .NET template, which is what this quick start guide demonstrates.

Setting up the Bot Framework Connector SDK .NET

This is a step-by-step guide to setting up the development environment to develop a bot in C# using the Bot Framework Connector SDK .NET template:

  1. Install the prerequisite software:
    1. You can download the community version of Visual Studio 2015 (latest update) for free at www.visualstudio.com.
    2. Update all VS extensions to their latest versions by navigating to Tools | ExtensionsandUpdates | Updates.
  2. Download and install the Bot Application template:
    1. Download the file from the direct download link, http://aka.ms/bf-bc-vstemplate.
    2. Save the ZIP file to your Visual Studio 2015 templates directory, which is traditionally in %USERPROFILE%DocumentsVisual Studio 2015TemplatesProjectTemplatesVisual C#.
  3. Open Visual Studio.

     

 

  1. Create a new C# project using the new Bot Application template:
  1. The template is a fully functional Echo Bot that takes the user's text utterance as input and returns it as output. In order to run, however, the following has to take place:
    1. The bot has to be registered with the Bot Connector.
    2. The AppId and AppPassword from the Bot Framework registration page have to be recorded in the project's web.config.
    3. The project needs to be published to the web.
    4. Use the Bot Framework emulator to test your bot application.

 The Bot Framework provides a channel emulator that lets you test calls to your bot as if they were being called by the Bot Framework cloud service. To install the Bot Framework emulator, download it from https://emulator.botframework.com/.

Once installed, you're ready to test, by starting your bot in Visual Studio using a browser as the application host:

 

Messages


Your bot can send rich text, emoticons, pictures, and cards to a user or group. Users can send rich text and pictures to your bot. You can specify the type of content your bot can handle in the Skype settings page for your bot:

Content

From user to bot

From bot to user

Description

Rich text

Including emoticons :)

Pictures

PNG, JPEG, or GIF up to 20 Mb

Video

Coming soon

MP4, AAC+h264 up to 15 Mb (approx. 1 minute), plus JPEG thumbnail

Cards

Basic format

Each Skype user is assigned a unique ID for your bot, which is sent along with the display name with every message:

{ 
  "text": "Hello (wave)", 
  "id": "1466182688092", 
  "type": "message/text", 
  "timestamp": "2016-06-17T16:58:08.74Z", 
  "channelId": "skype", 
  "serviceUrl": "https://apis.skype.com", 
  "from": { 
    "id": "29:2hJJkjmGn4ljB2X7YYEju-sgFwgvnISvE6G3abGde8ts", 
    "name": "Display Name" 
  }, 
  "conversation": { 
    "id": "29:2hJJkjmGn4ljB2X7YYEju-sgFwgvnISvE6G3abGde8ts" 
  }, 
  "recipient": { 
    "id": "28:29415286-5a43-4a00-9dc5-bcbc2ce1f59e", 
    "name": "Trivia Master" 
  } 
} 

The from field contains the unique user ID (prefixed by 29) and user Display Name. The recipient field contains the app ID (prefixed by 28, which indicates a bot in Skype) and the bot's display name. In Skype, your bot is addressed using the Bot Framework App ID (a GUID).

Note

You cannot currently use slash (/) commands as part of conversations with your bot. This is a reserved character in Skype.

Rich text

Users can communicate with the bot using rich text format as well. Users can make the chat text as bold if needed or a bot can communicate with the user and make certain words as bold. Most of the channels support all the text properties supported by the Microsoft Bot Framework.

Skype emoticons

Skype emoticons can be sent by using the emoticon keyword in parentheses:

{
  "text": "(heart)"
}

The output of the preceding code is as follows:

Note

If a user sends your bot an emoticon, it may include <ss> tags around the emoticon, which can be ignored; for example, <ss type="skype">(wave)</ss>. Sending Skype Mojis (short, expressive video clips) is not currently supported.

Welcome messages

To send a welcome message to a user, listen for the contactRelationUpdate activity. To send a welcome message to a group, listen for the conversationUpdate activity.

Pictures and videos

Let's check out, how pictures and videos are sent:

  • Pictures and videos are sent by adding attachments to a message
  • Pictures can be PNG, JPEG, or GIF up to 20 Mb
  • Videos can be MP4 or AAC+h264 up to 15 Mb (approx. 1 minute), plus JPEG thumbnails
 

Cards and buttons


Skype supports the following cards, which may have several properties and attachments. You can find information on how to use cards in the .NET SDK and Node.js SDK docs:

  • Hero card
  • Thumbnail card
  • Carousel card (with hero or thumbnail images)
  • Sign in card
  • Receipt card

Images sent to Skype cards need to be stored on an HTTPS endpoint. Skype cards do not currently support postBack actions.

Hero card

The hero card renders a title, subtitle, text, large image, and buttons:

The hero card provides a very flexible layout; for example, it might contain the following:

  • Image, title, subtitle, text, and three buttons
  • Title, subtitle, text, and five buttons
  • Title and six buttons
  • Image and six buttons

The following table illustrates the flexible layout of hero card:

Property

Type

Description

title

Rich text

Title of the card, maximum two lines.

subtitle

Rich text

Subtitle appears just below the title, maximum two lines.

text

Rich text

Text appears just below the subtitle; two, four, or six lines depending on whether the title and subtitle are specified.

images:[]

Array of images

Image displayed at top of the card; aspect ratio is 16:9.

buttons:[]

Array of action objects

Set of actions applicable to the current card: three buttons, up to a maximum of six (+two if no is image is shown, +one if the title or subtitle are not included, +two if the text is not included.)

tap

Action object

This action will be activated when the user taps on the card itself.

 

Thumbnail card

The thumbnail card renders a title, subtitle, text, small thumbnail image, and buttons:

The following table explains the properties of a thumbnail card:

Property

Type

Description

title

Rich text

Title of the card, maximum two lines.

subtitle

Rich text

Subtitle appears just below the title, maximum two lines.

text

Rich text

Text appears just below the subtitle: two, four, or six lines depending on whether the title and subtitle are specified.

images:[]

Array of images

Image displayed at top of the card; the image aspect ratio in a thumbnail card is 1:1.

buttons:[]

Array of action objects

Set of actions applicable to the current card; maximum three buttons.

tap

Action object

This action will be activated when the user taps on the card itself.

Carousel

The carousel card can be used to show a carousel of images and text, with associated action buttons:

Properties are the same as for the hero or thumbnail card.

Images

Images are scaled up or down in size while maintaining the aspect ratio to cover the image area, and then cropped from the center to achieve the image aspect ratio for the card.

Images should be stored on an HTTPS endpoint, up to 1024x1024, up to 1 MB in size, and in PNG or JPEG. The properties are explained in the following table:

Property

Type

Description

url

URL

URL to the image; Must be HTTPS.

alt

String

Accessible description of the image.

value

String

Action assigned to the image.

Buttons

Buttons are shown at the bottom of the card--in a single row if they fit, or stacked. Button text is always on a single line and will be trimmed if it is too long. If more buttons than can be supported by the card are included, they will not be shown.

Actions

The following table consists of properties, types and descriptions for actions:

Property

Type

Description

type

String

Required field. One of openURL (opens the given URL), imBack (posts a message in the chat to the bot that sent the card), call (Skype or phone number), showImage (for images only, displays the image),or signin (sign in card only).

title

String

Text description that appears on the button.

tap

Action object

Value depending on the type of action. For openURL it is a URL, for signin it is the URL to the authentication flow, for imBack it is a user defined string (which may contain hidden metadata for the bot for, example, <meta roomid='10'/>, for call it can be skype:skypeid or tel:telephone, and for showImage it is not required.

Sign in

The sign in card can be used to initiate an authentication flow with predefined images and title:

The following table illustrates the properties, types and descriptions of sign in:

Property

Type

Description

text

Rich text

Text appears just below the subtitle: two lines maximum.

buttons:[]

Array of action objects

Single button of type signin.

Receipt

The receipt card can be used to send a receipt. If the height of the card is too large, it is partially folded and a Show all action is shown to expand it:

The following table explains the properties, types and descriptions of receipt card:

Property

Type

Description

title

Rich text

Title of the card. Maximum two lines.

facts:[]

Array of fact key-value pairs

Fact key is left aligned, value is right aligned.

items:[]

Array of purchased objects

Properties: title (maximum two lines), subtitle (one line), text (up to six lines depending if the title, subtitle, and price are present), price, image (1:1 aspect ratio), tap.

total

Rich text

Title of the card. Maximum two lines.

tax

Rich text

Title of the card. Maximum two lines.

vat

Rich text

Title of the card. Maximum two lines.

items:[]

Rich text

Title of the card. Maximum two lines.

images:[]

Array of images

Image displayed at top of the card. Aspect ratio 16:9.

buttons:[]

Array of action objects

Set of actions applicable to the current card.

tap

Action object

This action will be activated when the user taps on the card itself.

 

Groups


A bot can be enabled for groups in the Skype settings for the bot. It can be added to a group chat in the same way as adding a participant to a chat. In a group, the bot will only receive messages directly addressed to it--for example, @YourBot This is the message. It will not receive other messages sent by group participants or notifications of users joining or leaving the group:

To enable a bot to be added to a group chat, you need to add this capability in Settings. Go to your bot Dashboard and edit the Skype channel:

 

Calling


You can build Skype bots that can receive and handle voice calls using the .NET SDK, Node.js SDK, or the Skype API.

Each time a Skype user places a call to your bot, the Skype bot platform will notify the bot using the calling WebHook you specify in Settings. In response, the bot can provide a set of basic actions called a workflow.

These are the supported actions:

  • Answer
  • Play prompt
  • Record audio
  • Speech to text
  • DTMF tones
  • Hang up

The Skype bot platform will execute the actions on the bot's behalf according to the workflow.

If the workflow is successful, Skype will post a result of the last action to your calling WebHook. For example, if the last action was to record an audio message, the result will be audio content.

During a voice call, your bot can decide, after each result, how to continue interaction with the Skype user.

Note

Skype bots with calling enabled are for preview only and cannot currently be published. To publish a bot in Skype, you will need to disable calling in the Skype settings for your bot and then set Published to enable in the bot dashboard. Bots can handle one-to-one calls, but not group calls.

 

 

Summary


In this chapter, we introduced the Microsoft Bot Framework and explained how it helps in the development of bots. Also, we have seen how to set up the development environment, emulator, and the tools needed for programming. This chapter is based on the thought that programming knowledge and experience grow best when they grow together. In the next chapter, we will introduce bot programming by building and locally deploying a simple Hello World bot application. You will get to know Visual Studio, C# .NET, the Bot Framework, and the related technologies along with all the steps for creating projects.

About the Author

  • Kishore Gaddam

    Kishore Gaddam is the CEO and a co-founder of Astrani Technologies and is recognized as an industry expert in mobile, cloud, and bot software development. He is a forward-thinking technology leader with 17+ years of international experience in building technology organizations, strategic planning, rolling out multiple platforms/products, IT program and project management, strategy and transformation (businesses, people, processes, and technologies), and growing business units across IoT, smart cities, NLP, AI, bots, cloud, robotics, mobile, healthcare, industrial automation, financial systems, retail, procurement (EPC), travel & leisure, logistics, manufacturing, and automotive domains.

    He is a champion of the technical pre-sales, architecture, and software development of enterprise Azure IoT/bot/web applications using cognitive services, microservices, Service Fabric, Azure IoT Hub, Stream Analytics, Cortana Intelligence Suit, Logic Apps, Notification Hubs, Big Data, Azure Web Apps, Azure App service, Azure API Apps, Application Insights, API Management, Machine Learning, Azure SQL databases, Cosmos DB, Data Factory, Data Lake, HD Insight, Redis Cache, Key Vault, and Azure Service Bus, and a champion of implementing DevOps using Azure, PowerShell scripts, ARM templates, and VSTS. He has huge experience in startup leadership, including building teams, and developing a Minimum Viable Product (MVP) with little to no supervision. Kishore is comfortable at all layers of the startup people stack, from individual contributor (software development, product management) to CxO.

    Kishore graduated in Technology Entrepreneurship from Stanford University, CA, and is a speaker at various conferences in the USA. Kishore is the author of the popular Microsoft Technologies blog. He has a love for mentoring and a passion for sharing new tools, programming languages, and technology trends at national conferences, regional code camps, local user groups, meetups, and hackathons.

    Browse publications by this author

Latest Reviews

(3 reviews total)
Excelente, gostei muito!!
Nivel básico, ejemplos de código obsoletos.
Very good book for learning bot programming and i found the chapters and concepts very useful

Recommended For You

Book Title
Unlock this book and the full library for FREE
Start free trial