References and further reading

1. The term foundation models lacks a widely agreed-upon definition. While some argue that these models must be large, trained using unsupervised or self-supervised learning, and serve as a basis for further fine-tuning, others disagree and suggest that the term is unnecessarily grandiose. Many experts outside of Stanford have pushed back against the term, citing concerns that it may be an attempt to coin a new term for something that does not need one. Instead, it may be more effective to use clearer, more descriptive language such as large pre-trained models or Large Self-Supervised Models (LSSMs), which more accurately capture the essence of these models without overemphasizing their importance.
2. On the Opportunities and Risks of Foundation Models: https://arxiv.org/abs/2108.07258.pdf.
3. On the Opportunities and Risks of Foundation Models: https://arxiv.org/pdf/2108.07258.pdf.
4. OpenAI Technical Report of GPT-4: https://cdn.openai.com/papers...