An explanation of the model selection rule in Figure 1

As we mention in the survey, we only include LLMs (larger than 10B) with publicly reported evaluation results in Figure 1. Excluding models with papers (because formal evaluation results are generally included in papers), models without papers contain Cohere, YaLM, Luminous, ChatGPT, Bard, and Vicuna. Among these models:

• Cohere, YaLM, Luminous, and ChatGPT are evaluated by HELM.
• Vicuna reports its results compared with other models at here.
• Bard is evaluated by paper 1, paper 2, and paper 3.

While some models do not comply with the criteria, they have played an important role in the development of large language models. We add them to the list and provide corresponding links for those who need them. We will continue collecting related models but will not be adding them until May 2023. Please let us know if you come across any models that meet the inclusion criteria. Thank you to everyone who provided suggestions for our paper.