Exploring Cutting-Edge LLMs with Vertex AI and PaLM 2

Maybe Chapter 7, The Generative AI Revolution with ChatGPT, made us think that OpenAI ChatGPT ended the tremendous progression of Large Language Models (LLMs) and Generative AI. Perhaps the incredible tools, such as GitHub Copilot, GPT-4, and OpenAI’s embedding models, stunned us into believing we had reached a limit.

Then, on May 23, 2023, Google released the PaLM 2 Technical Report, built on the earlier PaLM and Pathways papers. The content of these papers is mind-blowing! The architecture of PaLM improved both software and hardware performances with ingenious innovations.

We will begin the chapter by examining Pathways to understand PaLM. We will continue and look at the main features of PaLM (which stands for Pathways Language Model), a decoder-only, densely activated, and autoregressive model transformer model with 540 billion parameters trained on Google’s Pathways system. PaLM was trained on 780 billion...

Architecture

PaLM and PaLM2 were built on top of Pathways. Pathways is a Google technology that improves the efficiency of training LLMs through data parallelism, model parallelism, and execution-level parallelism.

We will begin with Pathways, the cornerstone of Google AI’s impressive achievements.

Pathways

The title of the Pathways paper may seem esoteric. Pathways: Asynchronous Distributed Dataflow by Barham et al. (2022) indeed appears like something you might want to avoid looking into. However, once you start reading the paper, you will be hooked!

If we look at some of the key features, we are somewhat stunned:

• Heterogeneous execution: Pathways can run programs on many devices, including TPUs, CPUs, and GPUs. This is a significant advance when assembling all the computing power we can get.
• Asynchronous execution: Pathways allows programs to be executed asynchronously. This might seem uninteresting, but PaLM will build on this technology...

Assistants

The structure of this section is similar to that of Chapter 7, The Generative AI Revolution with ChatGPT, and Chapter 8, Fine-Tuning OpenAI GPT Models.

We can view the similarity between Microsoft and Google assistants from two perspectives:

• The billion-dollar competition between these two giants is fierce, so they must produce products and services that match their technology rivalry.
• Standardization has sprung from necessity: they offer transformer Generative AI services in every application they sell, leading to an intuitive way to use them (prompts and parameters).

As we saw in Chapter 1, What Are Transformers?, transformers are General Purpose Technologies (GPTs) like electricity, computers, smartphones, and the internet. They have pervaded every domain and will increasingly progress.

However, each competing platform produces different results and performances that we must evaluate before building an application.

Transformer...

Vertex AI PaLM 2 API

The Vertex AI platform evolves continually. You are at the cutting edge!

The models that we use get updated from time to time. These updates can cause small differences in the results or how it works. The latest version might not be the most stable version. You will need to be on the watch and consult Google’s versioning documentation regularly: https://cloud.google.com/vertex-ai/docs/generative-ai/learn/model-versioning.

That is the price to pay to be on top of the market! However, the power of the models makes it worthwhile to explore PaLM 2. When the models stabilize, you will be far ahead of your competition. So, please fasten your seat belts, and let’s get the API rolling!

There are hundreds of known NLP tasks, and hundreds more mainstream users invent some every day. You cannot implement them all at once. Focus on understanding the tasks you explore in depth. You will then be able to adapt to new ones.

Open Google_Vertex_AI...

Fine-tuning

Fine-tuning becomes an option when the responses produce an acceptable result or when the prompt design does not meet expectations. Or not! In Chapter 11, Leveraging LLM Embeddings as an Alternative to Fine-Tuning, we saw that advanced prompt engineering leveraging OpenAI LLM Ada’s embeddings could produce good results.

So, what should we do? Prompt design by crafting good prompts with a ready-to-use model? Prompt engineering with an embedding model? Fine-tune a model to fit our needs?

Each of these choices comes with a cost. The best empirical method in computer science remains to:

• Rely on a reliable and optimized (volume, quality) evaluation dataset.
• Test different models and approaches. In this case, evaluate the outputs obtained through prompt design, engineering, and fine-tuning.
• Evaluate the risks and costs.

Like Amazon Web Services (AWS), Microsoft Azure, IBM Cloud, and others, Google Cloud provides a solid, simplified...

Summary

We saw that PaLM 2 could match and perhaps surpass other models in some fields. Google AI assistants, cloud tools, and APIs provide general-purpose functionalities. They resemble Microsoft Azure and OpenAI offers. However, the technology is quite different, as we saw with the architecture that led to PaLM 2. Also, many factors will influence the decision, and the outputs might vary from one task to another.

Before making a choice, extensive evaluations must be made with the right questions before working with Microsoft Azure, Google Cloud, and IBM Cloud, among others.

The chapter was divided into the four parts we will probably encounter when we explore new transformer models: architecture improvements, large language generative mainstream assistants, API implementation, and customization (fine-tuning or other methods).

We began the chapter by reviewing the many improvements in the architecture of LLM Generative AI models. Pathways opens the door to increased optimization...

Questions

1. Pathways is not a significant game-changer. (True/False)
2. PaLM 2 uses the same embedding matrices for input and output operations. (True/False)
3. PALM models have an encoder and a decoder stack. (True/False)
4. Google Workspace assistants contain Generative AI. (True/False)
5. All transformers are implemented for Generative AI. (True/False)
6. A generative LLM cannot perform discriminative tasks. (True/False)
7. Transformer models have reached their limits. (True/False)
8. PaLM 2 is the last Generative AI model Google will produce. (True/False)
9. Prompt engineering is an alternative to fine-tuning. (True/False)
10. Fine-tuning a transformer model does not require much dataset preparation. (True/False)

References

• Barham et al., 2022, Pathways: Asynchronous Distributed Dataflow for ML: https://arxiv.org/abs/2203.12533
• Chowdhery et al., 2022, PaLM: Scaling Language Modeling with Pathways: https://arxiv.org/pdf/2204.02311.pdf
• Anil et al., 2023, PaLM 2 Technical Report: https://arxiv.org/abs/2305.10403
• Google Cloud Vertex AI documentation: https://cloud.google.com/vertex-ai/docs/generative-ai/learn/overview

Further reading

• Generative AI repository (Google): https://github.com/GoogleCloudPlatform/generative-ai/#readme
• Roberto Gozalo-Brizuela, and Eduardo C. Garrido-Merchán, 2023, A survey of Generative AI Applications: https://arxiv.org/abs/2306.02781

Join our community on Discord

Join our community’s Discord space for discussions with the authors and other readers:

https://www.packt.link/Transformers