Abstract:
Transformer-based language models have undoubtedly become the dominant and favorite architecture for language generation of our time. However, although they provide impressive text quality, they tend to be hard to control. In the domain of image synthesis, on the other hand, Denoising Diffusion Models (DDM) are the dominant approach, shining with unprecedented quality and control. The application of DDMs to discrete domains like language remains a challenging open problem. This talk addresses this challenge head-on. First, we introduce Latent Diffusion for Language Generation that enables DDMs for text generation in the latent space of text auto-encoders, enabling the generation of fluent text through latent diffusion. Further, we utilize diffusion models to generate semantic proposals that guide autoregressive text decoders. The latter approach combines the fluency of autoregression with the plug-and-play control of diffusion. Through these works, we demonstrate how diffusion models can be adapted to language, opening new avenues for flexible and controllable language generation systems.
Biography:
Kilian Weinberger is a Professor in the Department of Computer Science at Cornell University. He received his Ph.D. from the University of Pennsylvania (Go Quakers!!) in Machine Learning under the supervision of Lawrence Saul and his undergraduate degree in Mathematics and Computing from the University of Oxford. During his career he has won several best paper awards at ICML (2004), CVPR (2004, 2017), AISTATS (2005), and KDD (2014, runner-up award). In 2011 he was awarded the Outstanding AAAI Senior Program Chair Award and in 2012 he received an NSF CAREER award. He is the recipient of the Daniel M Lazar ’29 Excellence in Teaching Award (2016) and the Ann S. Bowers Teaching and Advising Excellence Award (2024). As of 2024 he is an ACM and AAAI fellow, and in 2021 became a Blavatnik National Awards Finalists. He was elected co-Program Chair for ICML 2016 and for AAAI 2018, and has been on the ICML board since 2016. He became president of the ICML society in 2023. Since 2024 he has been a member of the Sloan Research Fellowships Selection Committee. Kilian Weinberger’s research focuses on Machine Learning and its applications. In particular, he has worked on learning under resource constraints, metric learning, AI in Science, computer vision, autonomous vehicles, Gaussian Processes, and deep learning. Before joining Cornell University, he was an Associate Professor at Washington University in St. Louis and before that he worked as a research scientist at Yahoo! Research in Santa Clara.
