Animating a Virtual Reality Experience

If you’re anything like me, animation has a special place in your heart. It’s not just cartoons on a screen; it’s also a way to bring ideas and stories to life. It’s a fantastic skill to have, particularly if you’re interested in the extended reality (XR) space.

We’ll start from the basics—writing a synopsis and a script for your animated sequence. We’ll then help you visualize your script by teaching you how to convert it into a storyboard. Once we’ve laid the groundwork, we’ll move on to some hands-on practice with Unity Timeline.

But remember—these are more than just skills. They’re tools that you can wield to give your animated sequences a unique style and flair. The processes we’ll cover are just guidelines—feel free to play around and make them your own. After all, creativity is at the heart of all great animation.

If we look back at...

Technical requirements

The complete source code for this chapter can be found at https://github.com/PacktPublishing/Enhancing-Virtual-Reality-Experiences-with-Unity-2022/tree/main/EnhancingVRExperiencesFullProject/Assets/_VRProjectAssets/Scenes/Chapter_13.

Understanding the animation pipeline

Animation pipeline is a term used to describe the process that animation studios and professional animators follow to create animated sequences. This process typically involves several stages that work in tandem, starting from an idea to a rough outline, then a basic design, and finally, a finished product. Almost every animated sequence that you have seen across a variety of mediums has been created using this process in one way, shape, or form, making it a fundamental part of the animation industry.

If you are familiar with video production or game design, you will find that many of the same concepts are used in the animation pipeline. As an animator, I use a variation of this process and many of the workflows I have developed in making my VR experiences. The idea of going from an idea to written words on paper to creating simple visuals, and then building out the experience, all stems from core principles I learned in my training as an animator...

Writing a synopsis

A synopsis for an animation sequence is a brief summary or outline that provides an overview of key elements and story beats in the sequence. It highlights the main characters, setting, conflict, and resolution to give a clear understanding of the narrative without going into too much detail. A well-written synopsis can help guide the development of the animation and keep the creative team focused on core ideas.

When creating a synopsis for an animation sequence, some questions to ask yourself might include the following:

• Who are the main characters, and what are their roles in the story?
• What is the primary conflict or challenge faced by the characters?
• How do the characters overcome this conflict or challenge?
• What is the setting or environment in which the sequence takes place?
• What is the emotional tone or mood of the sequence?

To create a synopsis for an animation sequence, follow these steps:

1. Identify the main characters...

Writing a script

A script for an animation sequence is a written document that provides a detailed blueprint of the story, including character dialogue, actions, and scene descriptions. The script serves as a guideline for animators, directors, and other creative team members involved in the production process.

When creating a script for an animation sequence, some questions to ask yourself might include the following:

• Does the script effectively convey the story and its themes?
• Are the characters well developed and engaging?
• Does the dialogue sound natural and authentic for each character?
• Are the pacing and timing appropriate for the intended audience and medium?
• Is the script visually engaging, with clear descriptions of actions and scene settings?

To create an animation script, follow these steps:

1. Develop your story idea: Begin by brainstorming and outlining the central premise, themes, and plot points of your story. This will serve as...

Drafting storyboards

A storyboard is a visual representation of an animation sequence, film, or TV show, presented in the form of a series of illustrations or images. It acts as a blueprint for the production, depicting the flow of the story, camera angles, character movements, and transitions between scenes. A storyboard helps the production team to plan and communicate their ideas effectively.

When creating a storyboard, some questions to ask yourself might include the following:

• Does the storyboard effectively convey the story and emotion of the scene?
• Are character expressions and movements clear and consistent?
• Are camera angles and shots varied and engaging?
• Does the pacing of the storyboard match the intended tone of the scene?
• Have all important story beats and visual elements been considered?

Here are some simple steps to follow when creating a storyboard:

1. Read and analyze the script: Familiarize yourself with the story, characters...

Creating a sequence animatic

A sequence animatic is a rough previsualization of an animation sequence, typically created using a series of storyboard images edited together with timing and basic movement and sometimes accompanied by voiceover, sound effects, or music. The purpose of an animatic is to visualize the narrative and pacing, plan out camera angles and movements, and make necessary adjustments before moving on to the final animation stage.

When creating a sequence animatic, some questions to ask yourself might include the following:

• Does the animatic effectively communicate the story and emotions of the scene?
• Are the pacing and timing of the sequence appropriate and engaging?
• Do camera angles and movements enhance the narrative and visual impact of the scene?
• Are there any elements of the scene that could be improved or changed for better clarity or storytelling?
• Does the animatic highlight key moments, expressions, or actions that are essential...

Adding audio

This is my favorite part of the animation process because adding audio to the scene early on allows me to create a better visual sequence. Often, our animated experiences are driven by sounds that immerse our ears in each event on the screen. Using audio, you can set the tone and communicate shifts and actions without having to use words or extensive visuals. For this animated sequence, I wanted to stay simple, sweet, and efficient. Therefore, I decided to set the audio track based on the tones and shifts that take place on the screen. This ultimately required me to choose three different soundtracks/audio files that could highlight those tone shifts.

To create an audio track (Figure 13.3) for our animated sequence, follow these steps:

1. In our animation timeline, create a new Audio Track.
2. Find the Dreams Of Her Best Friend audio file and place it in the Audio Track.
3. Set the start time to 0.
4. Set the duration to 17.46667.
5. Find the exploration_2B...

Previsualizing a sequence

Previsualization is an important step in this animation production process because this is where we convert our animatic from a collection of 2D images into a rough draft rendered in 3D. This is what I like to call the gray-box phase of development because we get to visualize what is intended to be seen in 3D. This process can take a considerable amount of time because we are carving out all the movement, but once it’s done, applying unique animation clips becomes much easier and takes less time. To save ourselves the hassle, we can create primitives for our character GOs to use as placeholders. When we actually have our character models and animation clips, we can just place those in the locations as easily as we did when we converted our gray box to a kitbash scene.

To create placeholders for our animated characters, follow these steps:

1. Create a GO and name it Main Character.
2. Set the Y position to 1.
3. Set the X position to -5.5 and...

Setting up cameras with Cinemachine

Cinemachine is a suite of camera tools for Unity, designed to help developers and creators build dynamic and complex camera systems for their games, animations, and interactive experiences without the need for extensive custom scripting. Cinemachine provides an intuitive and flexible system to control camera behavior, making it easier to create cinematic sequences, implement camera transitions, and manage multiple cameras within a scene. Cinemachine has two main components, as follows:

• Virtual Cameras: Cinemachine uses virtual cameras that can be placed throughout your scene to control the view and framing. These virtual cameras can have different priorities, allowing you to switch between them based on specific conditions or events.
• Cinemachine Brain: The Cinemachine Brain component is responsible for managing and blending the virtual cameras in your scene. It is attached to the main camera in your Unity project and acts as the central...

Downloading 3D characters

Before we start downloading a whole bunch of different 3D assets to replace all our primitive models, the first thing we need to do is break down the actual assets and needs we require. This is a systematic approach to streamline our process so that we can be efficient and save on space and time within our development process. Going back to what our actual script requires, we have a main character, two dancers, some standard background characters that we’ll call non-playable characters or NPCs, and our monster.

Instead of creating all these characters individually from scratch, we could use an asset library called Mixamo, which has characters and animations that we can utilize within our project. Looking at our project requirements, we’ll need a variety of animations, but we’ll also need specific characters that have unique elements. For our main character, we’ll use the character named James from the Mixamo character library...

Downloading animations from Mixamo

Even though we were able to download 3D models of characters from Mixamo, the true power of Mixamo is its extensive library of professionally curated animated sequences. Mixamo has been around for over a decade, and many creators and developers utilize this library to streamline their process of adding animation to their games and experiences. Just like the Unity Asset Store, Mixamo is a great resource for finding a variety of different animated clips that we could add to our characters in our scenes to enhance the animated experiences that we are creating. More importantly, this will save us a lot of time and allow us to have laser-focused searches for animations that have already been created. This is great because we don’t have to imagine what it’s going to look like; we could just utilize the Mixamo platform to test and prototype the animated sequences before we add them to our scene. While Mixamo is a great resource for 3D characters...

Setting up animated characters

Now that we have our 3D character FBX models in our Unity project, as well as the .fbx animation files from Mixamo, we can set them up using a humanoid character rig that is predefined in Unity and apply the animation files to the models in our scenes.

To set up the 3D characters to be used in our scenes, follow these steps:

1. Go to our 3D characters folder in the Project tab.
2. Choose the James 3D model.
3. In Inspector, go to the Rig tab (Figure 13.8).
4. Set the Animation Type dropdown menu to Humanoid.
5. Set the Avatar Definition dropdown menu to Create From This Model.
6. Click Apply to set up the rig 3D model with the humanoid animation system that Unity has.
7. Repeat steps 2–6 for the Michelle model, the Sporty Granny model, the X Bot model, and the Y Bot model.

Figure 13.8 – 3D model character rigging setup

Now that we have all of our models completed, you may notice that the models...

Adding animation clips to our timeline

Now that we have imported our 3D models, we need to add the animated files we downloaded from Mixamo. This process is slightly different from importing 3D model characters into Unity, as we need to focus on the animations rather than the materials included in the .fbx files we import. We will follow the same process we did for the 3D models by creating custom humanoid rigs for the animations we intend to use on our characters in Unity. Then, we will move to the Animation tab to make them work efficiently.

To set up our animation files for the animated scene, follow these steps:

1. In the Character Animation folder, select the breathing idle animation.
2. Set the Animation Type dropdown menu to Humanoid.
3. Set the Avatar Definition dropdown menu to Create from this Model.
4. Click Apply to save the changes.
5. Go to the Animation tab.
6. Set Animation compression to Optimal.
7. Enable Loop time.
8. Click Apply to save the changes...

Adding VFX

So, we have everything we need to make our animated experience come to life in VR. The last thing we’re going to do is really make it pop with explosions and VFX. These are the things that make the difference between having something that’s enjoyable and having something that’s memorable. If things such as explosions and particle beams are happening in front of you in VR, you know you’re taking that immersive experience to the next level. We can make our own VFX or utilize the Unity Asset Store to drag and drop prefabs that have pre-made VFX into our scenes. We’ll start by adding a particle beam to our monster so that it ups the ante of its destruction capabilities. Then, we can add a portal effect to show that the monster is coming out of some ethereal, crazy sci-fi origin place. Lastly, we’ll add some explosions and fires to show the destructive damage to the actual city from the monster’s rampage.

To add fire breath...

Adding background characters

An easy addition that we could quickly make is background characters. Even though we already have our animated sequence done, having background characters can liven up the environment even more. It’s great to have a main cast, but just having people in the background who are talking and taking up space can make the environment feel a lot more immersive and fuller as opposed to having a big open space that feels lifeless. We can revisit the People_Prototype prefab that we made and add audio sequences and animations, as well as replace the 3D model with the X Bot and Y Bot models that we downloaded. The best part about these models is that we can switch up the materials on them, giving them a variety of different looks.

To replace our People_Prototype prefab with a 3D model, follow these steps:

1. In the Prefabs folder, double-click the People_Prototype prefab to open it.
2. Find the X Bot 3D model and add it as a child of the People_Prototype...

Summary

In this chapter, we went through the full animation pipeline workflow from start to finish. Hopefully, by the end of this, you have a better understanding of how animation works and how it can be created. If you are interested in pursuing more animation in Unity, it is the perfect tool to get started and integrate into more projects. This is just the tip of the iceberg because the animation system in Unity is so robust, as we have seen thus far. The power of real-time rendering is that you can utilize animation in many different ways that you otherwise wouldn’t be able to in other programs such as Blender or Cinema 4D.

Just as our VR experiences are human-centered, the animation system and the way you create animation are interaction-centered, which in turn is human-centered as well. This is something to keep in mind as we craft interesting animated experiences, especially for VR because the utilization of these animations can be used across a variety of different...