Chapter 8.  Querying Structured Data in a Natural Way

In the previous chapter, we learned how we could use the current context to extend our knowledge on a certain topic. Throughout this chapter, we will continue with the Knowledge APIs. More specifically, we will learn how to explore relationships between academic papers and journals. We will see how we can interpret natural language queries, and get query expressions. Using these expressions, we will learn how to find academic entities. The next part will focus more on how to set up this kind of service on your own. At the end, we will look into the QnA Maker, to see how we can create FAQ services from existing content.

When we have completed this chapter, we will have covered the following topics:

• Interpreting natural language user queries using the Academic API

• Assisting the user with queries, using auto-complete features

• Using said queries to retrieve academic entities

• Calculating the distribution of academic entities from the queries

• Hosting...

Tapping into academic content using the Academic API

Microsoft Academic Graph (MAG) is a knowledge base for web-scale, heterogeneous entity graphs. Entities model scholarly activities, containing information such as field of study, author(s), institution, and more.

Data contained in MAG is indexed from the Bing web index. As this is continuously indexed, the data is always up to date.

Using the Academic API, we can tap into this knowledge base. Combining search suggestions, research paper graph search results, and histogram distributions, the API enables a knowledge-driven and interactive dialog.

When a user searches for research papers, the API can provide query completion. It may suggest queries based on the input. With a complete query, we can evaluate a query expression. This will retrieve a set of matching paper entities from the knowledge base.

Setting up an example project

To be able to test the Academic API, we want to create a new example project. Create this from the MVVM template...

Creating the backend using the Knowledge Exploration Service

The Knowledge Exploration Service (KES) is, in some ways, the backend for the Academic API. It allows us to build a compressed index from structured data, authoring grammar to interpret natural language.

To get started with KES, we need to install the service locally.

Note

To download the KES installer, go to https://www.microsoft.com/en-us/download/details.aspx?id=51488 .

With the installation comes some example data, which we will use.

The steps required to have a working service are as follows:

1. Define a schema.

2. Generate data.

3. Build the index.

4. Author grammar.

5. Compile grammar.

6. Host service.

Defining attributes

The schema file defines the attribute structure in our domain. When we previously discussed the Academic API, we saw a list of different entity attributes, which we could retrieve through the queries. This is defined in a schema.

If you open the file, Academic.schema, in the Example folder found where KES is installed, you will see...

Answering FAQs using QnA Maker

QnA Maker allows us to use existing frequently asked questions (FAQ) to create a bot that answers these questions. We can generate a knowledge base from existing FAQs, and train a model from it.

To get started, head over to https://qnamaker.ai. Log on or register, by clicking Sign in, in the top right corner. This will present you with the following screen:

If no services have been created yet, the list will be empty.

Creating a knowledge base from frequently asked questions

If no services have been created, we can create one by clicking on the Create new service tab. This will present us with the following screen:

Enter a useful name for the service. This is also where we enter the baseline FAQs to use. This can either be in the form of one or more URLs, or a file containing questions and answer pairs. For our example, we will be generating a knowledge base from the URL.

Note

If you do not have any FAQ to use, you can use the following from Microsoft: https...

Summary

Throughout this chapter, we have learned about the Academic API and Knowledge Exploration Service. We looked at how to interpret natural language queries to get query expressions for evaluation. Through this evaluation, we have retrieved academic papers from the Microsoft Academic Graph knowledge base. From there, we have learned how to set up the Knowledge Exploration service itself, going from schemas all the way to deploying it to a Microsoft Azure cloud service. In the end, we learned how to set up a simple QnA Maker service.

In the next chapter, we will move into the Search APIs, learning how to utilize the different search APIs offered by Bing.