A working group focusing on the technical challenges of training at scale on several hundred GPUs and how to make the best use of the (large) compute budget we have.
Note that participating in this working group does not imply that you will have direct access to the supercomputer since there are additional (quite strong) national restrictions on the access to this machine (see some details in the section on access to compute here). It does mean however that you will participate in the discussion on these aspects.
Sean Narenthiran, Stas Bekman, Thomas Wolf, Teven Le Scao, Morgan Funtowicz, Yozh (justheuristic), Max Ryabinin, Jean-Baptiste Cordonnier, Thijs Vogels, Akash Mehra, Antoine Bosselut, Samyam Rajbhandari, Gérard Dupont, Minjia Zhang, guillaume Alleon, Christophe Cerisara, James Briggs, Uğur Ünal, Harshit Pandey, Urmish Thakker, Sushil Bharati, Myriam Peyrounette, Hatim Bourfoune, Hadrien de Dompsure, Kamel Guerda, Siddharth Karamcheti, Laurel Orr, Espoir Murhabazi, Thomas Wang, Luca Moschella, Francois Courteille, Kari Briski, Meriem Bendris, Mohammad Shoeybi, Bryan Catanzaro, Jared Casper, Joey Conway, Sylvain Jeaugey, Camille Parisel, Deepak Narayanan,Bertrand Cabot, Rémi Lacroix, Sampo Pyysalo, Pierre-François Lavallée, Jeff Rasley, Olatunji Ruwase, Pawan Sasanka Ammanamanchi, Vikas Raunak, M Ali Jauhar, Arun Raja, Alberto Bégué, Dimitri Lozeve, Michael Z Wang, Zenodia Charpy, Shaden Smith
Yozh (jheuristic), Max Ryabinin