Creating Custom OpenAI Gym Environments - CARLA Driving Simulator

In the first chapter, we looked at the various categories of learning environments available in the OpenAI Gym environment catalog. We then explored the list of environments and their nomenclature in Chapter 5, Implementing your First Learning Agent – Solving the Mountain Car problem, as well as a sneak peek into some of them. We also developed our agents to solve the Mountain Car and Cart Pole problems, and a few Atari game environments. By now, then, you should have a good understanding of the various environment types and flavors that are available with OpenAI Gym. Most often, once we learn how to develop our own intelligent agents, we want to use that knowledge and skill to develop intelligent agents to solve new problems, problems that we already face, or even problems that are of interest to us. For...

Understanding the anatomy of Gym environments

Any Gym-compatible environment should subclass the gym.Env class and implement the reset and step methods and the observation_space and action_space properties and attributes. There is also the opportunity to implement other, optional methods that can add additional functionality to our custom environments. The following table lists and describes the other methods available:

Method	Functionality description
observation_space	The shape and type of the observations returned by the environment.
action_space	The shape and type of the actions accepted by the environment.
reset()	Routines to reset the environment at the start or end of an episode.
step(...)	Routines that calculate the necessary information to advance the environment, simulation, or game to the next step. The routine includes applying the chosen action...

Creating an OpenAI Gym-compatible CARLA driving simulator environment

CARLA is a driving simulator environment built on top of the UnrealEngine4 game engine with more realistic rendering compared to some of its competitors. You can read more about the CARLA simulator on their official website at https://carla.org. In this section, we will look into how we can create a custom OpenAI Gym-compatible car driving environment to train our learning agents. This is a fairly complex environment and requires a GPU to run—which is unlike other Gym environments we have seen so far. Once you understand how to create a custom environment interface that is Gym-compatible for CARLA, you may well have enough information to develop interfaces for any of your own custom environments, no matter how complex they might be.

The latest version of CARLA is CARLA 0.8.2. While most (if not all)...

Summary

In this chapter, we went through a custom Gym environment implementation step-by-step, starting with a template that laid out the bare-bones structure of an OpenAI Gym environment that provided all of the necessary interfaces to the agents. We also looked at how to register a custom environment implementation in the Gym registry so that we can use the familiar gym.make(ENV_NAME) command to create an instance of an existing environment. We then looked at how to create a Gym-compatible environment implementation for the UnrealEngine based on the open-source driving simulator, CARLA. We then quickly walked through the steps required to install and run CARLA and then started implementing the CarlaEnv class piece-by-piece, carefully covering all the important details involved in implementing custom environments compatible with OpenAI Gym.

In the next chapter, we will build...