As the current LLM research community pushes towards massive scale, this workshop takes a complementary approach by focusing on the under-explored small-scale regime, where scale encompasses compute, data, and model size. We ask: to what extent is scale necessary, and how far can we push toward smaller settings while maintaining competitive performance and enabling scientifically meaningful discoveries?
Beyond transferring insights from small- to large-scale settings, we emphasize the intrinsic value of understanding small-scale regimes. Practically, studying small-scale limits can yield more efficient algorithms and system designs. Small-scale settings also enable controlled, systematic experimentation with rapid iterations, thereby facilitating scientific progress.
This workshop aims to highlight the methods, opportunities, and insights enabled by small-scale experimentation, and to foster discussion on how such approaches can broaden participation and accelerate progress in LLM research.
Date: Oct 9, 2026
Location: TBD
(9:00 - 9:10) Opening Remarks
(9:10 - 10:40) Invited Talks
TBD
TBD
(10:40 - 11:00) Coffee break
(11:00 - 12:00) Contributed Talks and Demos (4 x 15min)
(12:00 - 13:30) Lunch break
(13:30 - 14:30) Panel Discussion
(14:30 - 16:00) Invited Talks
TBD
TBD
(16:00 - 18:00) Poster Session & Closing Remarks
Contact: colm2026-moss-workshop [at] googlegroups [dot] com