Developing a RandWireNN model from scratch

We discussed EfficientNets in Chapter 3, Deep CNN Architectures, where we explored the idea of finding the best model architecture instead of specifying it manually. RandWireNNs, or randomly wired neural networks, as the name suggests, are built on a similar concept. In this section, we will study and build our own RandWireNN model using PyTorch.

Understanding RandWireNNs

First, a random graph generation algorithm is used to generate a random graph with a predefined number of nodes. This graph is converted into a neural network by a few definitions being imposed on it, such as the following:

• Directed: The graph is restricted to be a directed graph, and the direction of edge is considered to be the direction of data flow in the equivalent neural network.
• Aggregation: Multiple incoming edges to a node (or neuron) are aggregated by weighted sum, where the weights are learnable.
• Transformation: Inside each node of this graph...