Adding Sound, Music, and Video in 3D Game Development with Microsoft Silverlight 3: Part 1

Exclusive offer: get 50% off this eBook here
3D Game Development with Microsoft Silverlight 3: Beginner's Guide

3D Game Development with Microsoft Silverlight 3: Beginner's Guide — Save 50%

A practical guide to creating real-time responsive online 3D games in Silverlight 3 using C#, XBAP WPF, XAML, Balder, and Farseer Physics Engine

$29.99    $15.00
by Gastón C. Hillar | October 2009 | Beginner's Guides Web Development Web Graphics & Video

In this article by Gastón C. Hillar we will add sounds, music and videos to the game. By reading this article and following the exercises, we will learn how to:

  • Add background music to the game
  • Generate sounds associated to the game events
  • Take advantage of multiple channel audio
  • Control the reproduction of concurrent sounds
  • Organize the media elements for the game
  • Prepare our media elements for their use in Silverlight applications
  • Take advantage of Silverlight 3 performance enhancements for videos
  • Reproduce videos combined with animated projections

A game needs sound, music and video. It has to offer the player attractive background music. It must also generate sounds associated with certain game events. When a spaceship shoots a laser beam, a sound must accompany this action. Reproducing videos showing high-quality previously rendered animations is a good idea during transitions between one stage and the next.

Hear the UFOs coming

So far, we have worked with 3D scenes showing 3D models with textures and different kinds of lights. We took advantage of C# object-oriented capabilities and we animated 3D models and moved the cameras. We have read values from many different input devices and we added physics, artificial intelligence, amazing effects, gauges, statistics, skill levels, environments, and stages. However, the game does not use the speakers at all because there is no background music and there are no in-game sounds. Thus, we have to sort this issue out. Modern games use videos to dazzle the player before starting each new stage. They use amazing sound eff ects and music custom prepared for the game by renowned artists. How can we add videos, music, and sounds in Silverlight?

We can do this by taking advantage of the powerful multimedia classes offered by Silverlight 3. However, as a game uses more multimedia resources than other simpler applications, we must be careful to avoid including unnecessary resources in the files that must be downloaded before starting the application.

Time for action – installing tools to manipulate videos

The 3D digital artists used Blender to create an introductory video showing a high quality rendered animation for five seconds. They took advantage of Blender's animation creation features, as shown in the following screenshot:

Adding Sound, Music, and Video in 3D Game Development with Microsoft Silverlight 3: Part 1

A spaceship flies in a starry universe for a few seconds. Then, the camera navigates through the stars.

Your project manager wants you to add this video as an introduction to the game. However, as the video file is in AVI (Audio Video Interleave) format and Silverlight 3 does not support this format, you have to convert the video to an appropriate format.

The creation of video animations for a game is very complex and requires specialist skills. We are going to simplify this process by using an existing video.

First, we must download and install an additional tool that will help us in converting an existing video to the most appropriate file formats used in Silverlight 3:

The necessary tools will depend on the applications the digital artists use to create the videos. However, we will be using some tools that will work fine with our examples.

Download one of the following files:

parceApplication's name

Download link

File name

Description

Expression Encoder 2

http://www.microsoft.com/expression/try-it/try-it-v2.aspx

Encoder_Trial_en.exe

It is a commercial tool, but the trial offers a free fully functional version for 30 days. This tool will enable us to encode videos to the appropriate format to use in Silverlight 3.

Expression Encoder 3

http://www.microsoft.com/expression/try-it

Encoder_Trial_en.exe

It is the newest trial version of the aforementioned commercial tool.

  1. Run the installers and follow the steps to complete the installation wizards.
  2. If you installed Expression Encoder 2, download and install its Service Pack 1.
  3. The download link for it is http://www.microsoft.com/expression/tryit/try-it-v2.aspx#encodersp1 file name—EncoderV2SP1_en.exe.
  4. Once you have installed one of the versions of Expression Encoder, you will be able to load and encode many video files in different file formats, as shown in the following screenshot:
  5. Adding Sound, Music, and Video in 3D Game Development with Microsoft Silverlight 3: Part 1

3D Game Development with Microsoft Silverlight 3: Beginner's Guide A practical guide to creating real-time responsive online 3D games in Silverlight 3 using C#, XBAP WPF, XAML, Balder, and Farseer Physics Engine
Published: September 2009
eBook Price: $29.99
Book Price: $49.99
See more
Select your format and quantity:

What just happened?

We installed Expression Encoder. Now we have the necessary tool to convert existing video clipart to the most appropriate file formats to use in Silverlight 3.

Time for action – preparing a video to use it in Silverlight

Now, we are going to use Expression Encoder to convert the video from an AVI format to an HD 720p Intranet (High Definition, 1280X720 pixels) encoding profile, compatible with Silverlight 3:

  1. Save or copy the original video file (introduction_HD.avi) in a new folder (C:Silverlight3DInvaders3DMedia).
  2. Start Expression Encoder.
  3. Select File | New Job.
  4. Now, select File | Import…. Browse to the folder that holds the video file (C:Silverlight3DInvaders3DMedia) and select the file introduction_HD.avi to import. Then, click on Open. Expression Encoder will analyze the file for a few seconds and then, the first frame for the video will appear on the preview area.
  5. Expand Profile and select HD 720p Intranet from the Video combo box. This step defines the desired video profile for the encoding process. The Video Profile options will display the output video's width, height, and aspect ratio, among other parameter values, as shown in the following screenshot:
  6. Adding Sound, Music, and Video in 3D Game Development with Microsoft Silverlight 3: Part 1

  7. Click on the Output tab, expand Job output and click on the Browse for output folder button (…) on the right side of the Directory text box. Browse to the folder that holds the original video file (C:Silverlight3DInvaders3DMedia) and click on OK.
  8. Select File | Encode or click on the Encode button. Expression Blend will begin the encoding job and will display the overall progress as shown in the following screenshot:
  9. Adding Sound, Music, and Video in 3D Game Development with Microsoft Silverlight 3: Part 1
  10. After a few seconds (depending on the video length, resolution, and format), Expression Blend will show Ready in the Status column.
  11. Right-click on the item (C:Silverlight3DInvaders3DMediaintroduction_HD.avi) under Media Content and select Open File Location in the context menu that appears. A new Explorer window will appear showing the folder with a new sub-folder with the default user name as a prefix, the date and the time. Enter this sub-folder and move the new video file (introduction_HD.wmv) to the previously mentioned parent folder.
  12. Double-click on the new video file (introduction_HD.wmv) and watch it using your default media player (Windows Media Player or VLC Media Player, among others). Check whether the video quality, resolution, and reproduction are as good as expected, as shown in the following screenshot:
  13. Adding Sound, Music, and Video in 3D Game Development with Microsoft Silverlight 3: Part 1

What just happened?

You used Expression Blend to encode the original AVI video into a WMV (Windows Media Video) with an HD 720p Intranet encoding profile. Now, the video has a 1280X720 pixels resolution and it is compatible with Silverlight 3.

In this case, we created a video prepared for an application that runs on the Intranet because we want to test the game. However, we will have to choose a diff erent encoding profile according to the Internet bandwidth offered by the hosting service and the average download speed available for the game's potential players. The steps to encode the video using a different profile are the same as the ones used in the previously explained procedure. The only step that changes is the one in which we select the desired encoding profile.

 

Video formats supported in Silverlight 3

Silverlight 3 supports the video encodings shown in the following table:

Encoding name

Description and restrictions

None

Raw video

YV12

YCrCb(4:2:0)

RGBA

32-bits Red, Green, Blue and Alpha

WMV1

Windows Media Video 7

WMV2

Windows Media Video 8

WMV3

Windows Media Video 9

WMVA

Windows Media Video Advanced Profile (non-VC-1)

WMVC1

Windows Media Video Advanced Profile (VC-1)

H.264

(ITU-T H.264 / ISO MPEG-4 AVC)

H.264 and MP43 codecs; base main and high profiles; only progressive (non-interlaced) content and only 4:2:0 chroma sub-sampling profiles

Silverlight 3 does not support interlaced video content.

If we want to use a video with an encoding format that does not appear in the previously shown table, we will have to convert it to one of the supported formats.

Using free applications to convert video formats

Expression Encoder is not the only application capable of converting videos to the encoding profiles supported by Silverlight 3. We can also use many free or open source applications and several online services to convert videos to any of the previously shown formats.

One example of a free and open source audio and video converter is MediaCoder (http://mediacoder.sourceforge.net/).

One example of a free online service to convert audio and video from one format to other is Media Convert (http://media-convert.com/). The advantage of online services dedicated to converting video formats is that they do not require the installation of additional software. Their trade-off is that we have to upload the original video file and then download the converted output video file. If the video files are big, we will need a very fast Internet connection.

If you have to convert too many big files and you do not have a fast Internet connection, installing a free and open source video converter such as MediaEncoder is a good choice.

Time for action – reproducing videos

Now, we are going to reproduce the video converted to a WMV format with an HD 720p Intranet encoding profile, before starting the game:

  1. Open the 3DInvadersSilverlight project.
  2. Open 3DInvadersSilverlightTestPage.aspx (double-click on it in the Solution Explorer, found under 3DInvadersSilverlight.Web project) and enable hardware acceleration (GPU acceleration).
  3. Create a new sub-folder in the ClientBin folder in the3DInvadersSilverlight.Web project. Rename it to Media.
  4. Right-click on the previously mentioned folder and select Add | Existing item… from the context menu that appears.
  5. Go to the folder in which you have copied the previously encoded video in the WMV format (C:Silverlight3DInvaders3DMedia). Select the WMV file and click on Add. This way, the video file will be part of the web project, in the new Media folder, as shown in the following screenshot:
  6. Adding Sound, Music, and Video in 3D Game Development with Microsoft Silverlight 3: Part 1

  7. Open MainPage.xaml (double-click on it in the Solution Explorer) and insert the following lines of code before the line that defines the cnvMainScreen Canvas:
    <Canvas x:Name="cnvVideo" Visibility="Collapsed" >
    <MediaElement x:Name="medIntroduction" Width="1366"
    Height="768" AutoPlay="False"
    CacheMode="BitmapCache" Stretch="UniformToFill"
    Source="Media/introduction_HD.wmv"
    MediaEnded="medIntroduction_MediaEnded"/>
    </Canvas>
  8. Add the following lines of code to program the event handler for the MediaElement's MediaEnded event (this code will start the game when the video finishes):
    private void medIntroduction_MediaEnded(object sender,
    RoutedEventArgs e)
    {
    StartGame();
    }
  9. Add the following private method to play the introductory video:
    private void PlayIntroductoryVideo()
    {
    // Show the Canvas that contains the MediaElement as a child
    cnvVideo.Visibility = Visibility.Visible;
    // Play the video
    medIntroduction.Play();
    }
  10. Replace the code in the transitionSB_Completed method with the following. (We want to play the introductory video instead of starting the game, once the transition finishes.):
    //StartGame();
    PlayIntroductoryVideo();
  11. Build and run the solution. Click on the butt on and the video will start its reproduction after the transition effect, as shown in the following screenshot:
  12. Adding Sound, Music, and Video in 3D Game Development with Microsoft Silverlight 3: Part 1

3D Game Development with Microsoft Silverlight 3: Beginner's Guide A practical guide to creating real-time responsive online 3D games in Silverlight 3 using C#, XBAP WPF, XAML, Balder, and Farseer Physics Engine
Published: September 2009
eBook Price: $29.99
Book Price: $49.99
See more
Select your format and quantity:

What just happened?

Now, the application shows an introductory video before starting the game. However, your project manager wants you to add some kind of 3D effect to the video. Players must understand that they are going to play a 3D game.

Locating videos in a related web site

First, we had to convert the original video to an encoding profile compatible with Silverlight 3. Then, we added it to a new Media sub-folder in the related website ClientBin folder. This way, we do not embed the large video file in the application's .xap file. Adding Sound, Music, and Video

The .xap file's size determines the time needed to download the Silverlight application. We are using the application's website to hold the media files. This way, we avoid generating a huge .xap file and the application downloads the media files on-demand.

The Source property uses a relative Uri (Media/introduction_HD.wmv) because the root folder for the .xap application is the ClientBin folder and the video is located in ClientBin/Media.

The AutoPlay property was set to False because we did not want the video to begin its reproduction until the transition ended.

Stretching videos

The video file prepared for Silverlight 3 uses a resolution of 1280X720 pixels. However, our game was prepared for 1366X768 pixels. Therefore, we created added a Canvas (cnvVideo) with a MediaElement instance (medIntroduction) as a child, using XAML code.

We defined the reproduction area to be 1366X768 pixels and we assigned the UniformToFill value to the Stretch property. Thus, the MediaElement resizes the original 1280X720 pixels video to fill its dimensions (1366X768 pixels) while preserving the video's native aspect ratio.

The following diagram shows the results of using the four possible values in the Stretch property with the same original video:

Adding Sound, Music, and Video in 3D Game Development with Microsoft Silverlight 3: Part 1

The following table explains the results of using the previously mentioned values:

Stretch value

Description

Aspect ratio

None

The video preserves its original size.

Preserved.

Uniform

The video is resized to fit in the destination dimensions.

Preserved.

UniformToFill

The video is resized to fill the destination dimensions. The video content that does not fit in the destination rectangle is clipped.

Preserved.

Fill

The video is resized to fill the destination dimensions.

Not preserved.

Taking advantage of GPU acceleration to scale videos

We configured the web project to start the Silverlight plug-in with hardware acceleration capabilities. As we are using the UniformToFill value for the Stretch property, assigning the BitmapCache value to the MediaElement's CacheMode property, Silverlight will try to perform the resize operation using the GPU.

The GPU acceleration will work only when the MediaElement has to show the video in a different size. If the value assigned to the Stretch property is None, the video size will not change and there will be no GPU assistance during the reproduction process.

First, we made cnvVideo visible and then we started reproducing the video calling medIntroduction's Play method, in the PlayIntroductoryVideo method.

As we wanted the game to start once the video ended, we programmed the MediaEnded event handler with the following line:

StartGame();

Time for action – applying projections

We want players to understand that they are going to play a 3D game. Hence, we will add a plane projection to the Canvas that contains the video (the MediaElement instance):

  1. Stay in the 3DInvadersSilverlight project.
  2. Open MainPage.xaml and insert the following lines of code after the line that defines the medIntroduction MediaElement:
    <Canvas.Projection>
    <PlaneProjection RotationX="-40" RotationY="15" RotationZ="-6"
    LocalOffsetX="-70" LocalOffsetY="-105" />
    </Canvas.Projection>
  3. Build and run the solution. Click on the button and the video will start its reproduction after the transition effect. However, this time, it will be displayed projected using a perspective transform, as shown in the following screenshot:

    Adding Sound, Music, and Video in 3D Game Development with Microsoft Silverlight 3: Part 1

What just happened?

Your project manager is amazed! The 3D digital artists are astonished! You could add a 3D perspective to the 2D video in just a few seconds. Now, the introductory video is even more attractive.

We added a PlaneProjection instance to the Canvas (cnvVideo) that contains the MediaElement (medIntroduction). Then, we assigned the following values to its properties:

  • RotationX: -40
  • RotationY: 15
  • RotationZ: -6
  • LocalOffsetX: -70
  • LocalOffsetY: -105

The RotationX, RotationY, and RotationZ properties specify the number of degrees to rotate the Canvas in the space. The LocalOffsetX and LocalOffsetY properties specify the distance the Canvas is translated along each axis of the Canvas' plane.

We can apply a perspective transform to any UIElement by setting its Projection property to a PlaneProjection instance. Then, we can assign the desired values to the PlaneProjection's properties.

PlaneProjection is a subclass of the Projection class. The last one allows you to describe how to project a 2D object in the 3D space using perspective transforms.

>> Continue Reading Adding Sound, Music, and Video in 3D Game Development with Microsoft Silverlight 3: Part 2

[ 1 | 2 ]

If you have read this article you may be interested to view :

 

About the Author :


Gastón C. Hillar

Gastón C. Hillar has been working with computers since he was eight. He began programming with the legendary Texas TI-99/4A and Commodore 64 home computers in the early 80s.
He has a Bachelor degree in Computer Science from which he graduated with honors, and an MBA from which he graduated with an outstanding thesis. Now, he is an independent IT consultant and a freelance author always looking for new adventures around the world.

To date he’s written more than 40 books in Spanish, and for Packt Publishing has written “C# 2008 and 2005 Threaded Programming: Beginner's Guide”. He usually writes articles for Spanish magazines Mundo Linux, Solo Programadores and Resistor.
He contributes to Dr. Dobb's Go Parallel programming portal http://www.ddj.com/go-parallel/ and he is a guest blogger at Intel Software Network http://software.intel.com

Gastón C. Hillar is the author of "Microsoft Silverlight 4 and SharePoint 2010 Integration".

Books From Packt

Blender 3D 2.49 Incredible Machines
Blender 3D 2.49 Incredible Machines

jQuery 1.3 with PHP
jQuery 1.3 with PHP

Symfony 1.3 Web Application Development
Symfony 1.3 Web Application Development

Zend Framework 1.8 Web Application Development
Zend Framework 1.8 Web Application Development

JBoss Tools 3 Developers Guide
JBoss Tools 3 Developers Guide

Seam 2.x Web Development
Seam 2.x Web Development

WordPress Plugin Development: Beginner's Guide
WordPress Plugin Development: Beginner's Guide

Scratch 1.4: Beginner’s Guide
Scratch 1.4: Beginner’s Guide

 

Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software