Chapter 2. Getting the User's Media

Obtaining a live video and audio feed from a user's webcam and microphone is the first step to creating a communication platform on WebRTC. This has traditionally been done through browser plugins, but we will use the getUserMedia API to do this all in JavaScript.

In this chapter, we will cover the following topics:

• Getting access to media devices

• Constraining the media stream

• Handling multiple devices

• Modifying the stream data

Getting access to media devices

There has been a long history behind trying to get media devices to the browser screen. Many have struggled with various Flash-based or other plugin-based solutions that required you to download and install something in your browser to be able to capture the user's camera. This is why the W3C decided to finally create a group to bring this functionality into the browser. The latest browsers now give the JavaScript access to the getUserMedia API, also known as the MediaStream API.

This API has a few key points of functionality:

• It provides a stream object that represents a real-time media stream, either in the form of audio or video

• It handles the selection of input devices when there are multiple cameras or microphones connected to the computer

• It provides security through user preferences and permissions that ask the user before a web page can start fetching a stream from a computer's device

Before we move any further, we should set a few standards about our coding...

Constraining the media stream

Now that we know how to get a stream from the browser, we will cover configuring this stream using the first parameter of the getUserMedia API. This parameter expects an object of keys and values telling the browser how to look for and process streams coming from the connected devices. The first options we will cover are simply turning on or off the video or audio streams:

navigator.getUserMedia({ video: false, audio: true }, function (stream) {
  // Now our stream does not contain any video!
});

When you add this stream to the <video> element, it will now not show any video coming from the camera. You can also do the opposite and get just a video feed and no audio. This is great while developing a WebRTC application when you do not want to listen to yourself talk all day!

Tip

Typically, you will be calling yourself a lot while developing the WebRTC applications. This creates the phenomenon known as audio feedback where you are being recorded through the microphone...

Handling multiple devices

In some cases, users may have more than one camera or microphone attached to their device. This is especially the case on mobile devices that often have a front-facing camera and a rear-facing one. In this case, you want to search through the available cameras or microphones and select the appropriate device for your user's needs. Fortunately, to do this, an API called MediaSourceTrack is exposed to the browser.

Note

On the other hand, since many of these APIs are still being created, not everything will be supported by all the browsers. This is especially the case with MediaSourceTrack, which is only supported in the latest version of Chrome at the time of writing this book.

With MediaSourceTrack, we can ask for a list of devices and select the one we need:

MediaStreamTrack.getSources(function(sources) {
    var audioSource = null;
    var videoSource = null;
    for (var i = 0; i < sources.length; ++i) {
      var source = sources[i];
      if(source.kind === ...

Modifying the media stream

We can take this project even further. Most image sharing applications today have some set of filters that you can apply to your images to make them look even cooler. This is also possible on the Web using CSS filters to provide different effects. We can add some CSS classes that apply different filters to our <canvas> element:

<style>
      .grayscale {
        -webkit-filter: grayscale(1);
        -moz-filter: grayscale(1);
        -ms-filter: grayscale(1);
        -o-filter: grayscale(1);
        filter: grayscale(1);
      }

      .sepia {
        -webkit-filter: sepia(1);
        -moz-filter: sepia(1);
        -ms-filter: sepia(1);
        -o-filter: sepia(1);
        filter: sepia(1);
      }

      .invert {
        -webkit-filter: invert(1);
        -moz-filter: invert(1);
        -ms-filter: invert(1);
        -o-filter: invert(1);
        filter: invert(1);
      }
    </style>

And, also some JavaScript to change the filter on click:

var...

Self-test questions

Q1. The browser will allow camera and microphone access even if the current page is opened as a file and not being served from a web server. True or false?

Q2. Which one of the following is not a correct browser prefix?

1. webkitGetUserMedia

2. mozGetUserMedia

3. blinkGetUserMedia

4. msGetUserMedia

Q3. The getUserMedia API will call the third argument as a function if an error happens while obtaining the camera or microphone stream. True or false?

Q4. Which one of the following does constraining the video stream not help with?

1. Securing the video stream

2. Saving processing power

3. Providing a good user experience

4. Saving bandwidth

Q5. The getUserMedia API can be combined with the Canvas API and CSS filters to add even more features to your application. True or false?

Summary

You should now have a firm grasp of the different ways to capture the microphone and camera streams. We also covered how to modify the constraints of the stream and select them from a set of devices. Our last project brought all of this together and added filters and image capture to create a unique photo booth application.

In this chapter, we covered how to get access to media devices, constrain the media stream, handle multiple devices, and modify the stream data.

The MediaStream specification is a constantly moving target. There are already plans to add a wealth of new features to the API enabling even more intriguing applications built on the Web. Always keep up with the latest specifications and support from the different browser vendors while developing WebRTC applications.

Although it may not seem like much, these techniques will help us in the upcoming chapters. Then knowing how to modify the input stream to ensure your WebRTC application is performing correctly will be the...