In this chapter, we will explore the world of machine learning through the following topics:
Predicting actions with an N-Gram predictor
Improving the predictor: Hierarchical N-Gram
Learning to use a Naïve Bayes classifier
Learning to use decision trees
Learning to use reinforcement
Learning to use artificial neural networks
In this chapter, we will explore the field of machine learning. This is a very extensive and intrinsic field in which even AAA titles have a hard time due to the amount of time that the techniques require for fine-tuning and experimentation.
However, the recipes that are contained in this chapter will give us a great head start in our endeavor to learn and apply machine-learning techniques to our games. They are used in several different ways, but the one we usually appreciate the most is difficulty adjustment.
Finally, you are advised to complement the recipes with the reading of more formal books on the subject, in order to gain theoretical insights that lie beyond the scope of this chapter.
Predicting actions is a great way to give players a challenge by going from random selection to selection based on past actions. One way to implement learning is by using probabilities in order to predict what the player will do next, and that's what an N-Gram predictor does.
To predict the next choice, N-Gram predictors hold a record of the probabilities of making a decision (which is usually a move), given all combinations of choices for the previous n moves.
This recipe makes use of general types. It is recommended that we have at least a basic understanding of how they work because it's critical that we use them well.
The first thing to do is implement a data type for holding the actions and their probabilities; we'll call it KeyDataRecord.
The KeyDataReconrd.cs file should look like this:
using System.Collections;
using System.Collections.Generic;
public class KeyDataRecord<T>
{
public Dictionary<T, int> counts;
...The N-Gram predictor can be improved by having a handler with several other predictors ranging from 1 to n, and obtaining the best possible action after comparing the best guess from each one of them.
We need to make some adjustments prior to implementing the hierarchical N-Gram predictor.
Add the following member function to the NGramPredictor class:
public int GetActionsNum(ref T[] actions)
{
string key = ArrToStrKey(ref actions);
if (!data.ContainsKey(key))
return 0;
return data[key].total;
}Just like the N-Gram predictor, building the hierarchical version takes a few steps:
Create the new class:
using System;
using System.Collections;
using System.Text;
public class HierarchicalNGramP<T>
{
public int threshold;
public NGramPredictor<T>[] predictors;
private int nValue;
}Implement the constructor for initializing member values:
public HierarchicalNGramP(int windowSize) ...
Learning to use examples could be hard even for humans. For example, given a list of examples for two sets of values, it's not always easy to see the connection between them. One way of solving this problem would be to classify one set of values and then give it a try, and that's where classifier algorithms come in handy.
Naïve Bayes classifiers are prediction algorithms for assigning labels to problem instances; they apply probability and Bayes' theorem with a strong-independence assumption between the variables to analyze. One of the key advantages of Bayes' classifiers is scalability.
Since it is hard to build a general classifier, we will build ours assuming that the inputs are positive- and negative-labeled examples. So, the first thing that we need to address is defining the labels that our classifier will handle using an enum data structure called NBCLabel:
public enum NBCLabel
{
POSITIVE,
NEGATIVE
}We already learned the power and flexibility of decision trees for adding a decision-making component to our game. Furthermore, we can also build them dynamically through supervised learning. That's why we're revisiting them in this chapter.
There are several algorithms for building decision trees that are suited for different uses such as prediction and classification. In our case, we'll explore decision-tree learning by implementing the ID3 algorithm.
Despite having built decision trees in a previous chapter, and the fact that they're based on the same principles as the ones that we will implement now, we will use different data types for our implementation needs in spite of the learning algorithm.
We will need two data types: one for the decision nodes and one for storing the examples to be learned.
The code for the DecisionNode data type is as follows:
using System.Collections.Generic;
public class DecisionNode
{
public string testValue;
...Imagine that we need to come up with an enemy that needs to select different actions over time as the player progresses through the game and his or her patterns change, or a game for training different types of pets that have free will to some extent.
For these types of tasks, we can use a series of techniques aimed at modeling learning based on experience. One of these algorithms is Q-learning, which will be implemented in this recipe.
Before delving into the main algorithm, it is necessary to have certain data structures implemented. We need to define a structure for game state, another for game actions, and a class for defining an instance of the problem. They can coexist in the same file.
The following is an example of the data structure for defining a game state:
public struct GameState
{
// TODO
// your state definition here
}Next is an example of the data structure for defining a game action:
public struct GameAction
{
// TODO
...Imagine a way to make an enemy or game system emulate the way the brain works. That's how neural networks operate. They are based on a neuron, we call it Perceptron, and the sum of several neurons; its inputs and outputs are what makes a neural network.
In this recipe, we will learn how to build a neural system, starting from Perceptron, all the way to joining them in order to create a network.
We will need a data type for handling raw input; this is called InputPerceptron:
public class InputPerceptron
{
public float input;
public float weight;
}We will implement two big classes. The first one is the implementation for the Perceptron data type, and the second one is the data type handling the neural network:
Implement a Perceptron class derived from the InputPerceptron class that was previously defined:
public class Perceptron : InputPerceptron
{
public InputPerceptron[] inputList;
public delegate float Threshold...Being a musician myself, this recipe is one that is close to my heart. Imagine a group of musicians with a base theme in mind. But, they've never played with each other, and as the song changes, they must adapt to the core tones with their instruments and their styles. The emulation of this adaptation is implemented using an algorithm called harmony search.
We will need to define an objective function as a delegate in order to set it up before calling the function.
We will now implement the algorithm in a class:
Create the HarmonySearch class:
using UnityEngine;
using System.Collections;
public class HarmonySearch : MonoBehaviour
{
}Define the public inputs that are to be tuned:
[Range(1, 100)] public int memorySize = 30; public int pitchNum; // consolidation rate [Range(0.1f, 0.99f)] public float consRate = 0.9f; // adjustment rate [Range(0.1f, 0.99f)] public float adjsRate = 0.7f; public float range = 0.05f; public...
© 2016 Packt Publishing Limited All Rights Reserved
