Keyon Vafa (Harvard): “Testing AI’s Implicit World Models”

Abstract:

Many of the robustness properties that are required for real-world applications of AI would be realized by a model that has understood the world. But it is unclear how to measure understanding, let alone how to define it. This talk will propose theoretically-grounded definitions and metrics that test for a model’s implicit understanding, or its world model. We will focus on two kinds of settings: one where implicit world models are tested behaviorally, and another that tests a model’s representation. These exercises demonstrate that models can make highly accurate predictions with incoherent world models, revealing their fragility.

Biography:

I’m a postdoctoral fellow at Harvard University as part of the Harvard Data Science Initiative, and I’m also an affiliate with LIDS at MIT. I work on developing new evaluation methodology for generative models in AI. I’m interested in capabilities that benchmarks can’t measure, like testing an LLM’s world model (i.e. whether it understands the world). While these ideas seem nebulous, my goal is to formalize them, derive evaluation methods, and implement them on models as they’re deployed.

I completed my PhD in computer science at Columbia University, where I was advised by David Blei. During my PhD, I was an NSF GRFP Fellow and Cheung-Kong Innovation Doctoral Fellow. I also interned at Google AI and Facebook AI Research. Upon graduating, I received the Morton B. Friedman Memorial Prize for excellence in engineering.

Zoom: https://upenn.zoom.us/j/95189835192, Passcode: 797599