Introducing Graph Convolutional Networks

The Graph Convolutional Network (GCN) architecture is the blueprint of what a GNN looks like. Introduced by Kipf and Welling in 2017 [1], it is based on the idea of creating an efficient variant of Convolutional Neural Networks (CNNs) applied to graphs. More accurately, it is an approximation of a graph convolution operation in graph signal processing. Thanks to its versatility and ease of use, the GCN has become the most popular GNN in scientific literature. More generally, it is the architecture of choice to create a solid baseline when dealing with graph data.

In this chapter, we’ll talk about the limitations of our previous vanilla GNN layer. This will help us to understand the motivation behind GCNs. We’ll detail how the GCN layer works and why it performs better than our solution. We’ll test this statement by implementing a GCN on the Cora and Facebook Page-Page datasets using PyTorch Geometric. This should improve...

Technical requirements

All the code examples from this chapter can be found on GitHub at https://github.com/PacktPublishing/Hands-On-Graph-Neural-Networks-Using-Python/tree/main/Chapter06.

Installation steps required to run the code on your local machine can be found in the Preface section of this book.

Designing the graph convolutional layer

First, let’s talk about a problem we did not anticipate in the previous chapter. Unlike tabular or image data, nodes do not always have the same number of neighbors. For instance, in Figure 6.1, node has 3 neighbors while node only has 1:

Figure 6.1 – Simple graph where nodes have different numbers of neighbors

However, if we look at our GNN layer, we don’t take into account this difference in the number of neighbors. Our layer consists of a simple sum without any normalization coefficient. Here is how we calculated the embedding of a node, :

Imagine that node has 1,000 neighbors and node only has 1: the embedding will have much larger values than . This is an issue because we want to compare these embeddings. How are we supposed to make meaningful comparisons when their values are so vastly different?

Fortunately, there is a simple solution: dividing the embedding...

Comparing graph convolutional and graph linear layers

In the previous chapter, our vanilla GNN outperformed the Node2Vec model, but how does it compare to a GCN? In this section, we will compare their performance on the Cora and Facebook Page-Page datasets.

Compared to the vanilla GNN, the main feature of the GCN is that it considers node degrees to weigh its features. Before the real implementation, let’s analyze the node degrees in both datasets. This information is relevant since it is directly linked to the performance of the GCN.

From what we know about this architecture, we expect it to perform better when node degrees vary greatly. If every node has the same number of neighbors, these architectures are equivalent: ():

1. We import the Planetoid class from PyTorch Geometric. To visualize the node degrees, we also import matplotlib and two additional classes: degree to get the number of neighbors of each node and Counter to count the number of nodes for each...

Predicting web traffic with node regression

In machine learning, regression refers to the prediction of continuous values. It is often contrasted with classification, where the goal is to find the correct categories (which are not continuous). In graph data, their counterparts are node classification and node regression. In this section, we will try to predict a continuous value instead of a categorical variable for each node.

The dataset we will use is the Wikipedia Network (GNU General Public License v3.0), introduced by Rozemberckzi et al. in 2019 [2]. It is composed of three page-page networks: chameleons (2,277 nodes and 31,421 edges), crocodiles (11,631 nodes and 170,918 edges), and squirrels (5,201 nodes and 198,493 edges). In these datasets, nodes represent articles and edges are mutual links between them. Node features reflect the presence of particular words in the articles. Finally, the goal is to predict the log average monthly traffic of December 2018.

In this section...

Summary

In this chapter, we improved our vanilla GNN layer to correctly normalize features. This enhancement introduced the GCN layer and smart normalization. We compared this new architecture to Node2Vec and our vanilla GNN on the Cora and Facebook Page-Page datasets. Thanks to this normalization process, the GCN obtained the highest accuracy scores by a large margin in both cases. Finally, we applied it to node regression with the Wikipedia Network and learned how to handle this new task.

In Chapter 7, Graph Attention Networks, we will go a step further by discriminating neighboring nodes based on their importance. We will see how to automatically weigh node features through a process called self-attention. This will improve our performance, as we will see by comparing it to the GCN architecture.

Further reading

• [1] T. N. Kipf and M. Welling, Semi-Supervised Classification with Graph Convolutional Networks. arXiv, 2016. DOI: 10.48550/ARXIV.1609.02907. Available: https://arxiv.org/abs/1609.02907.
• [2] B. Rozemberczki, C. Allen, and R. Sarkar, Multi-scale Attributed Node Embedding. arXiv, 2019. DOI: 10.48550/ARXIV.1909.13021. Available: https://arxiv.org/abs/1909.13021.