Understanding non-RL-based NAS

The core of NAS is about intelligently searching through different child architecture configurations by making decisions based on prior search experience to find the best child architecture in a non-random and non-brute-force way. The core of RL, on the other hand, involves utilizing a controller-based system to achieve that intelligence. Intelligent NAS can be achieved without using RL, and in this section, we will go through a simplified version of the progressive growing-from-scratch style of NAS without a controller and another competitive version of elimination from a complex fully defined NN macroarchitecture and microarchitecture.

Understanding path elimination-based NAS

First and foremost, differentiable architecture search (DARTS) is a method that extends the DAG search space defined in ENAS by removing the RL controller component. Instead of choosing previous nodes to connect to and choosing which operation to use for a node, all operations...