You're reading from Transformers for Natural Language Processing and Computer Vision - Third Edition

Product type Book

Published in Feb 2024

Publisher Packt

ISBN-13 9781805128724

Pages 728 pages

Edition 3rd Edition

Languages

Python

Concepts

Deep Learning

Author (1):

Denis Rothman

Table of Contents (24) Chapters

Preface

What Are Transformers?

Getting Started with the Architecture of the Transformer Model

Emergent vs Downstream Tasks: The Unseen Depths of Transformers

Advancements in Translations with Google Trax, Google Translate, and Gemini

Diving into Fine-Tuning through BERT

Pretraining a Transformer from Scratch through RoBERTa

The Generative AI Revolution with ChatGPT

Fine-Tuning OpenAI GPT Models

Shattering the Black Box with Interpretable Tools

Investigating the Role of Tokenizers in Shaping Transformer Models

Leveraging LLM Embeddings as an Alternative to Fine-Tuning

Toward Syntax-Free Semantic Role Labeling with ChatGPT and GPT-4

Summarization with T5 and ChatGPT

Exploring Cutting-Edge LLMs with Vertex AI and PaLM 2

Guarding the Giants: Mitigating Risks in Large Language Models

Beyond Text: Vision Transformers in the Dawn of Revolutionary AI

Transcending the Image-Text Boundary with Stable Diffusion

Hugging Face AutoTrain: Training Vision Models without Coding

On the Road to Functional AGI with HuggingGPT and its Peers

Beyond Human-Designed Prompts with Generative Ideation

Other Books You May Enjoy

Index

Appendix: Answers to the Questions

Foundation Models

Advanced large multipurpose transformer models represent such a paradigm change that they require a new name to describe them: Foundation Models. Accordingly, Stanford University created the Center for Research on Foundation Models (CRFM). In August 2021, the CRFM published a two-hundred-page paper (see the References section) written by over one hundred scientists and professionals: On the Opportunities and Risks of Foundation Models.

Foundation Models were not created by academia but by the big tech industry. Google invented the transformer model, leading to Google BERT, LaMBDA, PaLM 2, and more. Microsoft partnered with OpenAI to produce ChatGPT with GPT-4, and soon more.

Big tech had to find a better model to face the exponential increase of petabytes of data flowing into their data centers. Transformers were thus born out of necessity.

Let’s consider the evolution of LLMs to understand the need for industrialized AI models.

Transformers...