Part I: Defining text-to-image with Stable Diffusion
We will explore at a very low level the main Python files of the Keras version of Stable Diffusion, as shown in Figure 17.2. The complete code can be found at: https://github.com/keras-team/keras-cv/tree/master/keras_cv/models/stable_diffusion:
Figure 17.2: Stable Diffusion, Keras implementation
Figure 17.2 shows the Stable Diffusion architecture of the code we will explore that can be summed up in five phases:
- Text embedding.
- Random image creation.
- Stable Diffusion downsampling.
- Decoder upsampling.
- Output image.
The Keras Stable Diffusion code itself is only 500 lines long!
We will describe each function’s function, make a high-level mathematical representation, and find the Python classes that execute the process.
We will end the analysis by running a Keras notebook illustrating their talented compact code approach.
