Video Intelligence API

In the last chapter, we saw how to work with the Cloud Vision API. We then learned how to integrate this API with SmartExchange and get labels from the uploaded images, as well as administer an image before posting it as. Reply to a thread using Safe Search Annotations.. In this chapter, we are going to work with Video Intelligence API. We are going to let a user record a video from the app, and post it. Behind the scenes, we are going to upload this video to Video Intelligence API and get labels, as well as detect any explicit content. If the video is acceptable as per our forum guidelines, we post the video, else we reject the video.

The topics covered are:

• What is Video Intelligence API?
• Exploring Video Intelligence API
• Integrating Video Intelligence API with SmartExchange

Video Intelligence API

Google Video Intelligence API is one of the machine learning services exposed under the Cloud AI vertical. This service is used to analyze a video and its contents using machine learning models.

This service can:

• Gather insights from 20,000 labels
• Search video catalogues
• Distinguish scenes using shot detection

Some of the best use cases for this API are:

• Content recommendation
• Content moderation
• Contextual advertisements
• Searchable media archives

Using this API, we can detect the following:

• Label detection: From a given video, identify various items/things present in the video
• Explicit Content Detection: Detect adult content in a video, along with the position of occurance
• Shot Change detection: Detect scene changes in a video
• Video Transcription ^ALPHA: Transcribe the video content into English

...

Enabling API

Now that we have a project, and we have both API and service account keys, we will enable the required API and test our application. Navigate to the project home page (https://console.cloud.google.com/home/dashboard?project=smart-exchange-b10882). From the menu on the left-hand side, select APIs & Services | Library. Once we land on this page, search for Video Intelligence API and click on that card. Then, click on the Enable button. Your Enable API screen after enabling should look like this:

If you have not set up billing, you will be prompted to first set up billing before you continue.

This concludes our section on enabling the API. In the next section, we are going to explore the API.

Exploring Video Intelligence API

Please refer to the Setting up a Rest Client Section from Chapter 3, Cloud Vision API to set up a REST API client, before you continue. Now that we have all the required setup done, let's get started with exploring the API. In this section, we are going to explore only the label detection feature using REST API. You can explore other features by yourself in a similar fashion.

Label detection

We are going to start off with label detection. In the project folder that you downloaded along with this book (or you can download the same from https://github.com/PacktPublishing/Getting-Started-with-Google-Cloud-AI-Services) you will find a folder named Chapter 4 ; inside that folder you will find...

Integrating Video Intelligence API with SmartExchange

Now that we have seen what can be done using Video Intelligence API, let's actually integrate this into SmartExchange. The idea is that we will allow users to record a video and reply to the thread.

Using the Video Intelligence API service, we are going to fetch the video labels as well as check the video for any explicit content using the label detection and explicit content detection features. The final output of the uploaded video and its contents will look as shown here:

A simple and powerful way to protect the content on our sites, without any administration. So, let's get started with the implementation.

Solution design

To achieve our solution, we are going...

Deploying to Heroku

This step is optional, and you can do it if you are interested in deploying this app to Heroku. To continue with this section, you need to have set up a Heroku account and installed the Heroku Toolbelt. Once that is done, open a new Command Prompt or Terminal inside the smart-exchange-base folder, then run:

$ heroku login

This is a prompt for the Heroku credentials that you have signed up with. Next, let's get our application ready for deployment. Run this:

$ npm run build

Or, run this:

$ yarn build

This will run the required scripts to build the final dist folder, which will be deployed to Heroku. Once the build is completed, run the following:

$ git add -A
$ git commit -am "Video Intelligence API Integration Commit"

Let's push the code to Heroku. Run this:

$ git push heroku master

If everything goes well, the code will be uploaded to Heroku...

Summary

In this chapter, we went through the Google Video Intelligence API and we worked with two features of the API. Then, we set up the required code to integrate this API with the SmartExchange app. In the next chapter, we are going to work with the Cloud Speech API.