Chapter 4. Relation and Matching Networks Using TensorFlow

In the last chapter, we learned about prototypical networks and how variants of prototypical networks, such as Gaussian prototypical and semi-prototypical networks, are used for one-shot learning. We have seen how prototypical networks make use of embeddings to perform classification tasks.

In this chapter, we will learn about relation networks and matching networks. First, we will see what a relation network is and how it is used in one-shot, few-shot, and zero-shot learning settings, after which we will learn how to build a relation network using TensorFlow. Later in this chapter, we will learn about matching networks and how they are used in few-shot learning. We will also see different types of embedding functions used in matching networks. At the end of this chapter, we will see how to build matching networks in Tensorflow.

In this chapter, we will learn about the following:

• Relation networks
• Relation networks in one-shot, few-shot...

Relation networks

Now, we will see another interesting one-shot learning algorithm, called a relation network. It is one of the simplest and most efficient one-shot learning algorithms. We will explore how relation networks are used in one-shot, few-shot, and zero-shot learning settings.

Relation networks in one-shot learning

A relation network consists of two important functions: the embedding function, denoted by

, and the relation function, denoted by

. The embedding function is used for extracting the features from the input. If our input is an image, then we can use a convolutional network as our embedding function, which will give us the feature vectors/embeddings of an image. If our input is a text, then we can use LSTM networks to get the embeddings of the text.

As we know, in one-shot learning, we have only a single example per class. For example, let's say our support set contains three classes with one example per class. As shown in the following diagram, we have a support set containing...

Building relation networks using TensorFlow

The relation function is pretty simple, right? We will understand relation networks better by implementing one in TensorFlow.

You can also check the code available as a Jupyter Notebook with an explanation here: https://github.com/sudharsan13296/Hands-On-Meta-Learning-With-Python/blob/master/04.%20Relation%20and%20Matching%20Networks%20Using%20Tensorflow/4.5%20Building%20Relation%20Network%20Using%20Tensorflow.ipynb.

First, we import all of the required libraries:

import tensorflow as tf
import numpy as np

We will randomly generate our data points. Let's say we have two classes in our dataset; we will randomly generate some 1,000 data points for each of these classes:

classA = np.random.rand(1000,18)
ClassB = np.random.rand(1000,18)

We create our dataset by combining both of these classes:

data = np.vstack([classA, ClassB])

Now, we set the labels; we assign the 1label for classA and the 0label for classB:

label = np.vstack([np.ones((len(classA),1)),np.zeros...

Matching networks

Matching networks are yet another simple and efficient one-shot learning algorithm published by Google's DeepMind team. It can even produce labels for the unobserved class in the dataset.

Let's say we have a support set, S, containing K examples as

. When given a query point (a new unseen example),

, the matching network predicts the class of

by comparing it with the support set.

We can define this as

, where

is the parameterized neural network,

is the predicted class for the query point,

, and

is the support set.

will return the probability of

belonging to each of the classes in the dataset. Then, we select the class of

as the one that has the highest probability. But how does this work exactly? How is this probability computed? Let's us see that now.

The output,

, for the query point,

, can be predicted as follows:

 

Let's decipher this equation.

and

are the input and labels of the support set.

is the query input— the input to which we want to predict the label....

The architecture of matching networks

The overall flow of matching network is shown in the following diagram and it is different from the image we saw already. You can notice how the support set,

, and query set,

, are calculated through the embedding functions,

and

, respectively. As you can see, the embedding function, f, takes the query set along with the support set embeddings as input:

Matching networks in TensorFlow

Now, we will see how to build a matching network in TensorFlow step by step. We will see the final code at the end.

First, we import the libraries:

import tensorflow as tf
slim = tf.contrib.slim
rnn = tf.contrib.rnn

Now, we define a class called Matching_network, where we define our network:

class Matching_network():

We define the __init__ method, where we initialize all of the variables:

    def __init__(self, lr, n_way, k_shot, batch_size=32):

        #placeholder for support set
        self.support_set_image = tf.placeholder(tf.float32, [None, n_way * k_shot, 28, 28, 1])
        self.support_set_label = tf.placeholder(tf.int32, [None, n_way * k_shot, ])

        #placeholder for query set
        self.query_image = tf.placeholder(tf.float32, [None, 28, 28, 1])
        self.query_label = tf.placeholder(tf.int32, [None, ])

Let's say our support set and query set have images. Before feeding this raw image to the embedding function, first, we will extract the features...

Summary

In this chapter, we have learned how matching networks and relation networks are used in few-shot learning. We saw how a relation network learns the embeddings of the support and query sets and combines the embeddings and feeds them to the relation function to compute the relation score. We also saw how a matching network uses two different embedding functions to learn the embeddings of our support and query sets and how it predicts the class of the query set.

In the next chapter, we will learn how neural Turing machines and memory-augmented neural networks work by storing and retrieving information from the memory.

Questions

1. What are the different types of functions used in relation networks?
2. What is the operator Z in relation networks?
3. What is the relation function?
4. What is the loss function of relation networks?
5. What are the different types of embedding functions used in matching networks?
6. How is the class of the query point predicted in matching networks?

Further reading

• Matching networks: https://arxiv.org/pdf/1606.04080.pdf
• Relation networks: https://arxiv.org/pdf/1711.06025.pdf