The “BigScience” project originated from discussions in early 2021 between Thomas Wolf (HuggingFace), Stéphane Requena and Pierre-François Lavallee (respectively from GENCI and IDRIS, both institutions being behind the French supercomputer Jean Zay).
Very quickly, members of the science team of HuggingFace (Victor Sanh, Yacine Jernite and team) as well as members of the French academic and industrial AI and NLP research communities joined the discussions to further develop the project leading the grant application for 5 millions hours in February 2021.
<aside> 💡 About Jean Zay: During the first half of 2019, the Jean Zay computer was installed at IDRIS, national computing centre for the CNRS (Centre national de la recherche scientifique), with an impressive performance of 15.9 Pflop/s (15.9 million billion floating point operations per second). With an extension during 2020 the peak performance of 28 Pflops/s was achieved, opening new possibilites to extend beyond the classic usage modes of high performance computing to new usages in artificial intelligence.
</aside>
While BigScience quickly gathered a large French research community surrounding the public compute facility "Jean Zay", ranging from academic laboratories to startups/SMEs and larger industrial groups (identified as the founding members in the members page), it was also clear from the inception of the project that, to propose a more inclusive way to conduct open-research on these large scale artifacts, the international community should be included as soon as possibler and the project should be open and welcome the participants from a large and diverse set of nationalities, cultures, origins and research fields.
Following the early grant submission, the Founding members of the BigScience project thus started to open and extend the project to an international research community interested in studying and understand better the many research questions surrounding large language models as well as the challenges around creating and sharing such models and datasets for research purposes.
When the project reached more than 200 participants, the organisation of the project started to take shape and it was decided to adopt the structure of a research workshop which appeared as one of the most inclusive, flexible and research-community-focused structure for research collaboration.
The project is thus now defined as a one-year long research workshop with a set of collaborative tasks around the creation of a large multilingual dataset and a large multilingual language model as detailed in the Organization document.