Model-based methods
To begin, let's discuss the difference between the model-free approach that we have used in the book and model-based methods, including their strong and weak points and where they might be applicable.
Model-based versus model-free
In the The taxonomy of RL methods section in Chapter 4, The Cross-Entropy Method, we saw several different angles from which we can classify RL methods. We distinguished three main aspects:
- Value-based and policy-based
- On-policy and off-policy
- Model-free and model-based
There were enough examples of methods on both sides of the first and second categories, but all the methods that we have covered so far were 100% model-free. However, this doesn't mean that model-free methods are more important or better than their model-based counterparts. Historically, due to their sample efficiency, the model-based methods have been used in the robotics field and other industrial controls. This has also happened...
