Tatsu Hashimoto (Stanford University): “Statistical Methods for Trustworthy Language Modeling”

Presentation Abstract: 

Language models work well, but they are far from trustworthy. Major open questions remain on high-stakes issues such as detecting benchmark contamination, identifying LM-generated text, and reliably generating factually correct outputs. Addressing these challenges will require us to build more precise, reliable algorithms and evaluations that provide guarantees that we can trust.

Despite the complexity of these problems and the black-box nature of modern LLMs, I will discuss how in all three problems — benchmark contamination, watermarking, and factual correctness — there are surprising connections between classic statistical techniques and language modeling problems that lead to precise guarantees for identifying contamination, watermarking LM-generated text, and ensuring the correctness of LM outputs.

Speaker Bio: 

Tatsunori Hashimoto is an Assistant Professor in the Computer Science Department at Stanford University. He is a member of the statistical machine learning and natural language processing groups at Stanford and his work focuses on statistical approaches to improving and understanding language models. Work from his group spans many areas, including instruction-following and controllable language models, differentially private fine-tuning, and benchmarks for LM safety and capabilities. He received his Ph.D. at MIT under the supervision of Tommi Jaakkola and David Gifford, and is a Kavli fellow, a Sony and Amazon research award winner, and his work has been recognized with best paper awards at ICML and CHI.