- This event has passed.
Akshay Krishnamurthy (MSR): The coverage principle in language models: From pre-training to test-time scaling
Abstract:
Test-time compute has emerged as a new axis for scaling language model capabilities, yet we lack a principled understanding of this paradigm. What are the right algorithms and trade-offs for test-time scaling? What properties of the pre-trained model enable it? And can we better align pre-training recipes for test-time success? This talk addresses these questions through a unified lens of coverage. We first show that test-time scaling strategies like best-of-N sampling succeed if and only if the pre-trained model has coverage over high-quality responses. We then demonstrate that coverage, and hence best-of-N performance, can be improved through deliberate exploration, either purely at test time or via RL-style post-training. Finally, we ask why pre-training via next-token prediction yields models with good coverage in the first place. We uncover a rich theoretical landscape driven by an implicit bias of the next-token prediction objective, while also identifying a fundamental misalignment between next-token prediction and coverage, raising the possibility of future algorithmic innovations.
Biography:
I am a senior principal research manager at Microsoft Research, New York City. Previously, I spent two years as an assistant professor in the College of Information and Computer Sciences at the University of Massachusetts, Amherst and a year as a postdoctoral researcher at Microsoft Research, NYC. Before that, I completed my PhD in the Computer Science Department at Carnegie Mellon University, advised by Aarti Singh. I received my undergraduate degree in EECS at UC Berkeley.
My research interests are in machine learning and statistics. I am most excited about interactive learning, or learning settings that involve feedback-driven data collection. My recent interests revolve around decision making with limited feedback, including contextual bandits and reinforcement learning, and how these frameworks manifest in language modeling and generative AI.
