9

The Model Skeleton and Skin

Welcome to Chapter 9! In the previous chapter, we examined the glTF file format, its elements, and the relations between these elements. We also added a simple C++ class for reading data from an example file and displaying the character model on the screen.

In this chapter, we will explore the glTF format in more depth. Every character model has a skeleton, like a human. The skeleton is required to animate the parts of the character independently. You will learn how to extract the model’s skeleton and store the skeleton data in a tree structure.

Next, we will look at how to apply a skin to a character – the triangles that define the model. For the animations in the next chapter to appear correctly, the skin must follow the motion of the skeleton. Special attention must be paid to the joints between the bones of the model to ensure that the model’s skin behaves like human skin.

At the end of the chapter, we will look at...

Technical requirements

To follow along with this chapter, you will need the OpenGL and Vulkan renderer code from Chapter 8.

If we want to animate our game character model, we must extract the skeleton. This first step requires some work to complete, which we will cover in this chapter. As an example, we need to construct a tree structure of the model’s nodes and extract the so-called inverse bind matrices.

These skeletons are not spooky

If you think of a skeleton, the first picture in your mind will most probably be the one on the left side of Figure 9.1. But the type of skeleton we are talking about in this section is the one on the right side of the picture:

Figure 9.1: A human skeleton and the glTF example model skeleton

The skeleton in our example glTF model file looks surprisingly like a human skeleton. We can identify the hips, legs and feet, spine and neck, shoulders, and arms and hands.

Our first step on the way to creating the model’s skeleton is the creation of a hierarchical structure of all the nodes in the model. The hierarchy will let us propagate changes to one of the bones to the remaining parts of the skeleton connected to that bone.

Why do we create a node tree of the skeleton?

When you stretch out your left arm and raise it upward or to the side, you will automatically move all the parts of your arm with it. Your upper...

How (not) to apply a skin to a skeleton

To create a character for a game, we need to apply a body structure that fits the intended role in the game. For example, a male wizard has a different body than a female elf, and both are completely different to a human blacksmith. Therefore, the skin needs to reflect the amount of muscle and fat on the body of the model to appear plausible.

Naive model skinning

The naive way of applying a skin to a character skeleton is by using constant distances from the start and end of a node. This works if the entire model moves, but if individual nodes are rotated or translated, the character body will be distorted in an unwanted manner. In Figure 9.4, you can see the effect of the rotation of the middle and right nodes of a part of a functional character:

Figure 9.4. Naive idea of applying the skin to moving nodes gone wrong

Nodes are shown as blue arrows, vertices are red dots, and the skin is depicted by the red...

Implementing GPU-based skinning

The huge advantage of GPU-based calculations is the sheer amount of parallel execution achieved using shaders. We usually use only one CPU core to calculate the vertex position because multi-threading in code is complex and not easy to implement. In contrast, a GPU can run dozens or hundreds of shader instances in parallel, depending on the model and driver. The shader units are also specialized to do vertex and matrix operations, generating even more speed in the vertex position calculation.

Moving the joints and weights to the vertex shader

To move the calculation to the vertex shader, a new shader pair needs to be created. We can use the gltf.vert vertex shader and the gltf.frag fragment shader as the basis and copy the files to new files called gltf_gpu.vert and gltf_gpu.frag.

While the fragment shader can be used without changes, the vertex shader needs a couple of additions:

layout (location = 3) in vec4 aJointNum;
layout (location...

Identifying linear skinning problems

To get an idea of the problem, Figure 9.9 shows a simple box, twisted in the middle:

Figure 9.9: A noticeable volume loss in the middle when twisting the model

We can see that the twist leads to volume loss, a phenomenon we thought we had solved by using the joints affecting the vertices, and the weights of the joints per vertex. Apparently, something in the calculation is still going wrong.

Particularly on a sharp bend or twist, the linear interpolation may lead to wrong results. This is because linear interpolation uses the shortest path between the vertices:

Figure 9.10: Shortest path for linear interpolation using matrices

If we use quaternion interpolation instead of linear interpolation, the paths of the connection between the vertices will be located on an arc between the two locations, keeping the virtual volume of the model in this place:

Figure 9...

Summary

In this chapter, we explored the skeleton of the glTF model and different methods of applying the vertex skin to the character model.

First, we created a tree structure for the skeleton. This step is required for the vertex skinning process, as we need the transformation matrices of the nodes to alter the vertex positions properly.

Next, we extracted all the data elements from the glTF file required to apply the vertex skinning. The CPU-based skinning was done to show the basic principle of the process. Then, we switched to GPU-based vertex skinning, moving the calculations from the processor to the vertex shader. Using the GPU instead of the CPU leads to a huge performance boost, as the massive parallel shader calculation is much faster than our single CPU core.

Finally, we added dual quaternion vertex skinning as a GPU skinning variant. Using dual quaternions enables a better, volume-retaining transformation behavior than linear blending. The dual quaternion approach...

Practical sessions

Try out these ideas to get a deeper insight into vertex skinning:

• Implement dual quaternion skinning also on the CPU side. This is simpler than the GLSL variant because you can use the quaternion and dual quaternion data types of GLM in the code, and do not have to convert them to 2x4 matrices. Compare the timings with the normal CPU vertex skinning.
• Adjust the vector normals in the shader to follow the changes in the vertices. Right now, the vertex normals are copied unchanged to the fragment shader, and the normals are not altered when the model triangles are rotated. This leads to incorrect lighting on the model as the direction of the normal and the direction of the triangle no longer match. Hint: use the transpose of the inverse matrix.
• Clean up the renderers and remove the box model data. In the next chapter, we will fully concentrate on character animations, and the boxes will most probably obstruct the exploration of the animations.

...

Additional resources

• Introduction to dual numbers: https://en.wikipedia.org/wiki/Dual_number
• Dual quaternion skinning overview: https://users.cs.utah.edu/~ladislav/dq/index.html
• Dual quaternion skinning paper: https://users.cs.utah.edu/~ladislav/kavan08geometric/kavan08geometric.pdf
• Dual quaternion skinning with scale: http://rodolphe-vaillant.fr/entry/78/dual-quaternion-skinning-with-scale
• Dual quaternion bulge fix: http://rodolphe-vaillant.fr/entry/72/bulge-free-dual-quaternion-skinning-trick
• Extracting the Euler angles from a rotation matrix: https://en.wikipedia.org/wiki/Euler_angles#Conversion_to_other_orientation_representations