LMSS @ Cornell Tech: Aditi Raghunathan (CMU)

Learning Machines Seminar Series What: LMSS: Aditi Raghunathan (CMU) When: Thursday, October 23, 1:30-2:45 pm Where: Bloomberg 081, Bloomberg Center, Cornell Tech (map) The series is organized by Associate Professor Yoav Artzi and sponsored by Bloomberg. Pizza will be served at 1:15 p.m. "The missing laws of modern scaling" Modern scaling laws tell only part of the story. While larger models trained on ever more tokens continue to excel on static benchmarks, they overlook two essential dimensions of intelligence: adaptability and creativity. We uncover catastrophic overtraining, a surprising inversion of the “more data is better” intuition, where excessive pretraining reduces downstream adaptability. For example, OLMo-1B trained on 3T tokens fine-tunes worse than its 2.3T-token counterpart, defying expected scaling trends. Our analysis shows this arises from a systematic increase in the broad sensitivity of pretrained parameters to subsequent modifications. We then examine creativity through controlled algorithmic tasks that abstract open-ended reasoning. Next-token prediction fails at the far-sighted leaps necessary for creative synthesis, whereas multi-token training objectives overcome this limitation. Our results further show that injecting noise at the input layer can elicit randomness without sacrificing coherence, offering a simple path toward richer reasoning diversity. Building on these insights, we propose a scalable mode-conditioning framework that dynamically allocates test-time compute across diverse reasoning modes, achieving up to 8× efficiency gains on reasoning benchmarks such as AIME. BIO Aditi Raghunathan is an Assistant Professor of Computer Science at Carnegie Mellon University. Her work advances trustworthy AI by translating insights from the scientific study of frontier model failures into methods that make them robust and safe. She is a recipient of the NSF CAREER Award, Okawa Research Award, Schmidt AI2050 Early Career Fellowship, Google Research Scholar Award, Forbes 30 Under 30 recognition, Arthur Samuel Best Thesis Awards, and multiple PhD fellowships. Her work has also been recognized with an Outstanding Paper Award at ICML 2025 and several workshop paper awards.