You're reading from Transformers for Natural Language Processing - Second Edition

Product type Book

Published in Mar 2022

Publisher Packt

ISBN-13 9781803247335

Pages 602 pages

Edition 2nd Edition

Languages

Concepts

Mobile Application Development

Author (1):

Denis Rothman

Table of Contents (25) Chapters

Preface

1. What are Transformers?

2. Getting Started with the Architecture of the Transformer Model

3. Fine-Tuning BERT Models

4. Pretraining a RoBERTa Model from Scratch

5. Downstream NLP Tasks with Transformers

6. Machine Translation with the Transformer

7. The Rise of Suprahuman Transformers with GPT-3 Engines

8. Applying Transformers to Legal and Financial Documents for AI Text Summarization

9. Matching Tokenizers and Datasets

10. Semantic Role Labeling with BERT-Based Transformers

11. Let Your Data Do the Talking: Story, Questions, and Answers

12. Detecting Customer Emotions to Make Predictions

13. Analyzing Fake News with Transformers

14. Interpreting Black Box Transformer Models

15. From NLP to Task-Agnostic Transformer Models

16. The Emergence of Transformer-Driven Copilots

17. The Consolidation of Suprahuman Transformers with OpenAI’s ChatGPT and GPT-4

18. Other Books You May Enjoy

19. Index

Appendix I — Terminology of Transformer Models

1. Appendix II — Hardware Constraints for Transformer Models

2. Appendix III — Generic Text Completion with GPT-2

3. Appendix IV — Custom Text Completion with GPT-2

4. Appendix V — Answers to the Questions

Summarization with GPT-3

It was essential to understand the architecture of a T5 transformer. We will also see how GPT-3 engines behave on one of the texts. The goal is not to benchmark companies and models. The goal is for an Industry 4.0 AI Guru to have a broad knowledge of NLP.

First, go to https://openai.com/ and sign up and sign in.

Then go to the examples page and select Summarize for a 2nd grader:

Une image contenant texte Description générée automatiquement

Figure 8.8: GPT-3 examples page

A window will open, and we can enter our prompt.

We submit the text T of the corporate sample of the previous section to the GPT-3 model.

The prompt is P = E + T + S:

E tells the model to make the explanation simple:
My second grader asked me what this passage means:

The text T is the same as in the previous section and is in quotes:
"""The law regarding corporations prescribes that a corporation can be incorporated in the state of Montana to serve any lawful purpose...