Creating the network in PyTorch
Now that we’ve downloaded the CIFAR-10 dataset, split it into test and training data, and reshaped and rescaled it, we are ready to start building our VAE model. We’ll build on the example at https://github.com/lyeoni/pytorch-mnist-CVAE in this section; however, for our purposes, we will implement simpler VAE networks using MLP layers based on the original VAE paper, Auto-Encoding Variational Bayes5, and show how we adapt the PyTorch example to also allow for IAF modules in decoding.
In the original article, the authors propose two kinds of models for use in the VAE, both MLP feedforward networks: Gaussian and Bernoulli, with these names reflecting the probability distribution functions used in the MLP network outputs in their final layers.
Creating a Bernoulli MLP layer
The Bernoulli MLP can be used as the decoder of the network, generating the simulated image 
from the latent vector 
. The formula for the Bernoulli MLP is...
