Akari Asai (University of Washington): “Beyond Scaling: Frontiers of Retrieval-Augmented Language Models”

Abstract:

Large Language Models (LMs) have demonstrated remarkable capabilities by scaling up training data and model sizes. However, they continue to face critical challenges, including hallucinations and outdated knowledge, which particularly limit their reliability in expert domains such as scientific research and software development. In this talk, I will urge the necessity of moving beyond the traditional scaling of monolithic LMs and advocate for Augmented LMs—a new AI paradigm that designs, trains, and deploys LMs alongside complementary modules to address these limitations. Focusing on my research on Retrieval-Augmented LMs, one of the most impactful and widely adopted forms of Augmented LMs today, I will begin by presenting our systematic analyses of current LM shortcomings and demonstrate how Retrieval-Augmented LMs offer a more effective and efficient path forward. I will then discuss my work to establish new foundations for further reliability and efficiency by designing and training new LMs and retrieval systems to dynamically adapt to diverse inputs. Finally, I will demonstrate the real-world impact of such Retrieval-Augmented LMs through OpenScholar, our fully open Retrieval-Augmented LM designed to assist scientists in synthesizing scientific literature, now used by more than 25,000 researchers and practitioners worldwide. I will conclude by outlining my vision for the future of Augmented LMs, emphasizing advancements in their abilities to handle heterogeneous and diverse modalities, more efficient and effective integration with diverse components, and advancing evaluations with interdisciplinary collaboration.

Biography:

Akari Asai is a Ph.D. candidate in the Paul G. Allen School of Computer Science & Engineering at the University of Washington. Her research addresses the limitations of large language models (LMs) by developing advanced systems, such as Retrieval-Augmented LMs, and applying them to real-world challenges, including scientific research and underrepresented languages. Her contributions have received widespread recognition, including multiple paper awards at top NLP and ML conferences, the EECS Rising Stars 2022, and MIT Technology Review’s Innovators Under 35 Japan. She has also been honored with the IBM Global Fellowship and several industry grants. Akari actively engages with the research community as a co-organizer of high-impact tutorials and workshops, including the first tutorial on Retrieval-Augmented LMs at ACL 2023, as well as NAACL 2022 workshop on Multilingual Information Access and NAACL 2025 Workshop on Knowledge-Augmented NLP.

Zoom Link: https://upenn.zoom.us/j/95663463468