Exploring the scope of GPT-3

Even the most powerful transformers such as OpenAI GPT-3 have their limits. Let’s see how GPT-3 reacts to the word amoeboid, which is closer to a medical term than a mainstream word. We will need technical jargon in many projects. Matching datasets requires quality control of how a transformer organizes its dictionary and embeddings.

We humans can detect errors and correct somebody. For example, in this chapter, we explored the word amoeboid in the Controlling tokenized data section of this chapter.

Let’s first ask GPT-3 what amoeboid means:

Figure 9.4: Asking GPT-3 what “amoeboid” means

amoeboid (resembling an amoeba) is an adjective, yet GPT-3 states that it is a noun in the output:

A: Amoeboid is a noun which means "resembling an amoeba"

We then ask GPT-3 a more precise question and still obtain an incorrect answer:

Q: Is amoeboid a noun or an adjective?
A: Amoeboid is a noun.

...