Guarding the Giants: Mitigating Risks in Large Language Models

On May 16, 2023, Sam Altman, CEO of OpenAI, the owner of ChatGPT, addressed the Congress of the United States by saying, “Our goal is to demystify AI and hold accountable those new technologies and to avoid some of the mistakes of the past.” This statement shows that we must mitigate the risks in Large Language Models (LLMs).

Our journey up to this chapter in this book has answered the question of Chapter 1, What Are Transformers? – transformers are General-Purpose Technologies (GPTs). Through mainstream applications, they have become assistants in every domain: social media, productivity software (word processors, spreadsheets and slides), development copilots, and more.

AI is only one of the many GPTs, including electricity, nuclear energy, combustion engines, computer chips, and electronic connections. All these technologies have a point in common: it is impossible to imagine how they will...

The emergence of functional AGI

The increasing pervasiveness of transformer-driven AI in every domain for intellectual tasks will inevitably lead to a massive evolution of Foundation Models. Massive Multitask Language Understanding (MMLU) models will soon overtake LLMs.

Functional Artificial General Intelligence (AGI) will probably emerge in the future through necessity. AI is not conscious, sentient, or human in any sense. However, as shown in several NLP benchmarks, AI doesn’t need to be conscious to outperform humans in many fields.

To illustrate the emergence of functional AGI in this section, we will speculate on the future of LLM evaluations and controls, and how this may lead to AI replicants.

Let’s do the math:

• BIG-bench is an LLM evaluation platform: https://github.com/google/BIG-bench/blob/main/bigbench/benchmark_tasks/README.md.
• The platform contains 200+ NLP tasks.
• The Center for Research on Foundation Models (CRFM) at the...

Cutting-edge platform installation limitations

Cutting-edge platforms are continuously modifying, upgrading, and updating their applications, creating regular instabilities.

Let’s explore OpenAI’s installation on Google Colab on January 16, 2024, for any notebook:

#Importing openai
!pip install openai

Several packages are installed successfully but with specific versions:

equirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai) (3.7.1)
Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai) (1.7.0)
Collecting httpx<1,>=0.23.0 (from openai)
  Downloading httpx-0.26.0-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━...

Auto-BIG-bench

Will AI soon be able to evaluate itself? Let’s take a step forward into the future and see what is most probably coming.

Open Auto-BIG-bench.ipynb from this chapter’s folder in the repository. The program will feed GPT-4 a sample of 140+ BIG-bench tasks with a two-part prompt.

The first part contains the following instructions:

"1.Explain the following task
2.Provide an example
Solve it":

Note that the instructions do not require punctuation, only a whitespace.

The second part is the description of BIG-bench, for example:

Given a narrative, choose the most related proverb

GPT-4 will then:

1. Read the first part of the instructions.
2. Read the BIG-bench NLP task to be performed.
3. Create an example of the task.
4. Solve it.

This aspect is another step toward functional AGI. In the future, another AI model will probably evaluate and improve the response.

To illustrate this potential leap...

WandB

WandB has advanced AI tracking capabilities. Imagine the future. Imagine that one day, OpenAI GPT-4 can understand WandB tracking information on its activity, such as the Auto-Big-bench.ipynb notebook. Once AI can do that, the door to functional AGI is wide open!

Open WandB_Prompts_Quickstart.ipynb from the chapter’s repository.

The notebook is self-explanatory. You will need a WandB key and an OpenAI key, as we saw in Chapter 8, Fine-Tuning OpenAI GPT Models.

You can run the notebook and follow the instructions to see how WandB can track OpenAI and LangChain activity.

Let’s focus on the following cell:

tool_span.add_named_result({"input": "search: google founded in year"}, {"response": "1998"})
chain_span.add_named_result({"input": "calculate: 2023 - 1998"}, {"response": "25"})
llm_span.add_named_result({"input": "calculate: 2023 - 1998", "...

When will AI agents replicate?

In this section, GPT-4 demonstrates its ability to generate and explain code independently, beyond its “copilot” role. Microsoft Copilot and Google Colab Copilot help us write code. What if the AI agent behind the copilots doesn’t need us to replicate on their own? What if an AI agent’s role becomes pilot, not copilot? What if an organization creates a pipeline with sufficient machine power and data to weaponize an LLM for commercial, political, or military goals? This model could:

• Design and write a transformer model from scratch to replicate itself in many domains for an indefinite number of functions.
• Scrape data from any website to build a dataset for misinformation, disinformation, political influencing campaigns, and more ill-intentioned purposes.
• Deploy itself through the pipeline and enter an indefinite number of online forums or social media platforms, make comments on any website, and communicate...

Risk management

There is no order of risks of artificial intelligence in this section. Every risk can have damaging effects. Transformers that perform generative or discriminative tasks have flaws and weaknesses that must be addressed. These risks are inherited from LLM transformers, which are inherited from machine learning technology. The stochastic, random nature of machine learning has been transmitted from one generation of artificial intelligence to another.

This section contains seven critical risks related to LLMs, such as ChatGPT with GPT-4 and PaLM 2: hallucinations, risky emergent behavior, disinformation, influence operations, harmful content, privacy, cybersecurity, and memorization.

The limitations of this section are:

• Not all risks are covered.
• The examples of the risks and harms are designed to show the issues but they only explain why they must be banned. They do not tell us how to solve the issues.
• Research labs are working hard to...

Risk mitigation tools with RLHF and RAG

This section will take us from prompt design to advanced prompt engineering with some mitigation tools to get us started in this domain:

• RLHF

  You can organize Reinforcement Learning from Human Feedback (RLHF) beyond the process described in this section. The term may seem daunting, but you can organize this with a group of key users who can provide feedback on the responses of your system. Then, you can adapt the system accordingly and modify hyperparameters, parameters, datasets, and any aspect of the project before fine-tuning the model again or implementing RAG, for example.

• RAG

  This section implements a method of Retrieval-Augmented Generation (RAG) through a knowledge base. There are several possible approaches, such as the ones we implemented in Chapter 7, The Generative AI Revolution with ChatGPT, and Chapter 11, Leveraging LLM Embeddings as an Alternative to Fine-Tuning. A customized knowledge base...

Summary

Foundation Models offer many opportunities but come with critical risks that must be taken seriously. We saw how some of the best models on the market, such as ChatGPT, GPT-4, and Vertex AI PaLM 2, could stumble occasionally.

Hallucinations can lead to stating that an elephant landed on the moon. Or invent novels that don’t exist. Risky emergent behaviors and disinformation can damage the credibility of LLMs and harm others. Influence campaigns can disrupt the classical flow of information.

Before implementing cloud platform LLMs, we need to check the privacy policies and perform cybersecurity checks.

To mitigate the risks, we went through some of the possible tools. We added a rule base to the moderation model. A knowledge base can create a relatively closed ecosystem and limit open uncontrolled dialogs. The system can be steered with informative messages added to the prompt.

Finally, we saw that token management is an excellent way to control user...

Questions

1. It’s impossible to force ChatGPT to harass somebody. (True/False)
2. Hallucinations are only for humans. (True/False)
3. Privacy is taken seriously on the leading cloud platforms. (True/False)
4. APIs pose no risk. (True/False)
5. Harmful content can be filtered. (True/False)
6. A moderation model is 100% reliable. (True/False)
7. A rule base is useless when using LLMs. (True/False)
8. A knowledge base will make the transformer ecosystem more reliable. (True/False)
9. We cannot add information to a prompt. (True/False)
10. Prompt engineering requires more effort than prompt design. (True/False)

References

• GPT-4 Technical Report, OpenAI, March 27, 2023: https://arxiv.org/pdf/2303.08774.pdf
• Bommansani et al. 2023, Ecosystem Graphs: The Social Footprint of Foundation Models: https://arxiv.org/abs/2303.15772
• OpenAI research: https://openai.com/research
• Google Generative AI: https://github.com/GoogleCloudPlatform/generative-ai/blob/main/language/prompts/intro_prompt_design.ipynb
• BIG-bench platform: https://github.com/google/BIG-bench/blob/main/bigbench/benchmark_tasks/README.md

Further reading

• Anderljung et al., 2023, Frontier AI Regulation: Managing Emerging Risks to Public Safety: https://arxiv.org/abs/2307.03718
• GDPR compliance: https://gdpr.eu/
• SOC 2 compliance: https://www.imperva.com/learn/data-security/soc-2-compliance/
• Stanford Ecosystem Graphs, which attempts to have a solid inventory of Foundation Models and resources: https://crfm.stanford.edu/ecosystem-graphs/index.html?mode=home
• Stanford University’s Center for Research on Foundation Models: https://crfm.stanford.edu/

Join our community on Discord

Join our community’s Discord space for discussions with the authors and other readers:

https://www.packt.link/Transformers