Anqi “Angie” Liu (JHU): “Robust and Uncertainty-Aware Decision Making under Distribution Shifts”

Abstract:

Decision making tasks like contextual bandit and reinforcement learning often need to be conducted under data distribution shifts. For example, we may need to utilize off-policy data to evaluate a target policy and/or learn an optimal policy utilizing logged data. We may also need to deal with sim2real problem when there is a dynamics shift between training and testing environments. In this talk, I am going to introduce two threads of my work in the domain of robust decision making under distribution shifts. First, I will introduce distributionally robust off-policy evaluation and learning techniques that feature a more conservative uncertainty in the reward estimation component. This pessimistic reward estimation will benefit both off-policy evaluation and learning under various distribution shifts. Second, I will introduce our work in estimating model performance under distribution shift and off-dynamics reinforcement learning, where recognizing the underlying structures in distribution shift benefits model auditing and model adaptation. Especially, previous off-dynamics reinforcement learning methods can suffer from a lack of exploration, while we propose a novel model-based approach that better estimates the target dynamics, leveraging shared structures. Finally, I will survey some of our current work and future work in uncertainty-aware approaches to critical applications in large language models and health decision making.

Biography:

Anqi “Angie” Liu is an assistant professor in the Department of Computer Science at the Whiting School of Engineering and a member of the Data Science and AI Institute. She is broadly interested in developing principled machine learning algorithms for building more reliable, trustworthy, and human-compatible AI systems in the real world. Her research focuses on enabling the machine learning algorithms to be robust to the changing data and environments, to provide accurate and honest uncertainty estimates, and to consider human preferences and values in AI interactions. She is particularly interested in high-stake applications that concern the safety and societal impact of AI.

Liu develops, analyzes, and applies methods in statistical machine learning, deep learning, and sequential decision-making. One established line of her work is in distributionally robust learning under covariate shift. Her recent projects cover topics in different types of distribution shifts, active learning, safe exploration, off-policy learning, fair machine learning, semi-supervised learning, cost-sensitive classification, and hierarchical classification.

Zoom: https://upenn.zoom.us/j/92346171614