Fri, April 11, 9:00 AM
60 MINUTES
Discovering Agent-Centric Latent States in Theory and in Practice

A latent state can enable vastly better planning, exploration, and credit assignment by keeping task-relevant information while discarding distractions and irrelevant details. For example, in video games, there is a game-engine state which has all relevant information for the underlying dynamics. This tutorial will discuss how we can discover such a latent state in the real world directly from observations, and the kinds of latent states which are known to be discoverable. The tutorial discusses theoretical developments at a high-level, to explain the key pieces of understanding as well as their limitations. The tutorial will discuss where the state-of-the-art is experimentally, and what is currently ready for usage in real-world applications.

Alex Lamb

Senior researcher @ Microsoft Research

I am a senior researcher in the real-world reinforcement learning group at Microsoft Research, working under John Langford. I completed my PhD at the University of Montreal advised by Yoshua Bengio and was a recipient of the Twitch PhD Fellowship 2020. My research is on the intersection of developing new algorithms for machine learning and new applications. In the area of algorithms, I'm particularly interested in (1) making deep networks more modular and richly structured and (2) improving the generalization performance of deep networks, especially across shifting domains. I am particularly interested in techniques which use functional inspiration from the brain and psychology to improve performance on real tasks. In terms of applications of Machine Learning, my most recent work has been on historical Japanese documents and has resulted in KuroNet, a publicly released service which generates automatic analysis and annotations to make classical Japanese documents (more) understandable to readers of modern Japanese.