Emerging Applications in Generative AI

In the preceding chapters, we have examined a large number of applications using generative AI, from generating pictures and text to even music. However, this is a large and ever-expanding field; the number of publications on Google Scholar matching a search for "generative adversarial networks" is 27,200, of which 16,200 were published in 2020! This is astonishing for a field that essentially started in 2014, the exponential growth of which can also be appreciated on the Google n-gram viewer (Figure 13.1):

Figure 13.1: Google n-gram of "generative adversarial networks"

As we saw in this volume, generative adversarial networks are only one class of models in the broader field of generative AI, which also includes models such as variational autoencoders, BERT, and GPT-3. As a single book cannot hope to cover all of these areas, we conclude this volume with discussion of a number of emerging topics in this field...

Finding new drugs with generative models

One field that we have not covered in this volume in which generative AI is making a large impact is biotechnology research. We discuss two areas: drug discovery and predicting the structure of proteins.

Searching chemical space with generative molecular graph networks

At its base, a medicine – be it drugstore aspirin or an antibiotic prescribed by a doctor – is a chemical graph consisting of nodes (atoms) and edges (bonds) (Figure 13.2). Like the generative models used for textual data (Chapters 3, 9, and 10), graphs have the special property of not being fixed length. There are many ways to encode a graph, including a binary representation based on numeric codes for the individual fragments (Figure 13.2) and "SMILES" strings that are linearized representations of 3D molecules (Figure 13.3). You can probably appreciate that the number of potential features in a chemical graph is quite large; in fact, the number...

Solving partial differential equations with generative modeling

Another field in which deep learning in general, and generative learning in particular, have led to recent breakthroughs is the solution of partial differential equations (PDEs), a kind of mathematical model used for diverse applications including fluid dynamics, weather prediction, and understanding the behavior of physical systems. More formally, a PDE imposes some condition on the partial derivatives of a function, and the problem is to find a function that fulfills this condition. Usually some set of initial or boundary conditions is placed on the function to limit the search space within a particular grid. As an example, consider Burger's equation,⁸ which governs phenomena such as the speed of a fluid at a given position and time (Figure 13.8):

Where u is speed, t is time, x is a positional coordinate, and is the viscosity ("oiliness") of the fluid. If the viscosity is 0, this simplifies...

Few shot learning for creating videos from images

In prior chapters, we have seen how GANs can generate novel photorealistic images after being trained on a group of example photos. This technique can also be used to create variations of an image, either applying "filters" or new poses or angles of the base image. Extending this approach to its logical limit, could we create a "talking head" out of a single or a limited set of images? This problem is quite challenging – classical (or deep learning) approaches that apply "warping" transformations to a set of images create noticeable artifacts that degrade the realism of the output ^13,14. An alternative approach is to use generative models to sample potential angular and positional variations of the input images (Figure 13.11), as performed by Zakharov et al. in their paper Few Shot Adversarial Learning of Realistic Neural Talking Head Models.¹⁵

Figure 13.11: Generative architecture for...

Generating recipes with deep learning

A final example we will discuss is related to earlier examples in this book, on generating textual descriptions of images using GANs. A more complex version of this same problem is to generate a structured description of an image that has multiple components, such as the recipe for a food depicted in an image. This description is also more complex because it relies on a particular sequence of these components (instructions) in order to be coherent (Figure 13.12):

Figure 13.12: A recipe generated from an image of food¹⁷

As Figure 13.13 demonstrates, this "inverse cooking" problem has also been studied using generative models¹⁷ (Salvador et al.).

Figure 13.13: Architecture of a generative model for inverse cooking¹⁷

Like many of the examples we've seen in prior chapters, an "encoder" network receives an image as input, and then "decodes" using a sequence model into text representations...

Summary

In this chapter, we examined a number of emerging applications of generative models. One is in the field of biotechnology, where they can be used to create large collections of new potential drug structures. Also in the biotechnology field, generative models are used to create potential protein folding structures that can be used for computational drug discovery.

We explored how generative models can be used to solve mathematical problems, in particular PDEs, by mapping a set of boundary conditions of a fluid dynamics equation to a solution grid. We also examined a challenging problem of generating videos from a limited set of input images, and finally generating complex textual descriptions (components and sequences of instructions) from images of food to create recipes.

Well done for reaching the end of the book. As a final summary, let's recap everything we've learned:

• Chapter 1, An Introduction to Generative AI: "Drawing" Data from...

References

1. Kirkpatrick, P., & Ellis, C. (2004). Chemical space. Nature 432, 823. https://www.nature.com/articles/432823a
2. Villanueva, J.C. (2009, July 30). How Many Atoms Are There in the Universe? Universe Today. https://www.universetoday.com/36302/atoms-in-the-universe/#:~:text=At%20this%20level%2C%20it%20is,hundred%20thousand%20quadrillion%20vigintillion%20atoms
3. Based on a figure from Akutsu, T., & Nagamochi, H. (2013). Comparison and enumeration of chemical graphs. Computational and Structural Biotechnology Journal, Vol. 5 Issue 6
4. Gómez-Bombarelli R., Wei, J. N., Duvenaud, D., Hernández-Lobato, J. M., Sánchez-Lengeling, B., Sheberla, D., Aguilera-Iparraguirre, J., Hirzel, T. D., Adams, R. P., Aspuru-Guzik, A. Automatic chemical design using a data-driven continuous representation of molecules. ACS central science 2018, 4, 268-276.
5. Blaschke, T., Olivecrona, M., Engkvist, O., Bajorath, J., Chen, H. Application of generative autoencoder...

Why subscribe?

• Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
• Learn better with Skill Plans built especially for you
• Get a free eBook or video every month
• Fully searchable for easy access to vital information
• Copy and paste, print, and bookmark content

At www.Packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.