Vanessa Kosoy

The learning-theoretic research agenda is a programme for creating a mathematical theory of intelligent agents, and using this theory to solve the AI alignment problem. This programme draws on several areas of mathematics / theoretical computer science / decision theory, such as statistical learning, computational complexity, algorithmic information and imprecise probability. This talk will give you a small taste of what problems the agenda is concerned with, and how it might all come together to solve alignment.

Vanessa will also be doing a four-hour-long session for only the most pure mathy folks of the workshop

We will attempt to do a deep dive into the mathematical frameworks coming out of the learning-theoretic agenda, such as infra-Bayesianism, infra-Bayesian physicalism and the AIT intelligence measure. We will discuss Physicalist Superimitation (the alignment protocol previously known as PreDCA). Depending on time and participant preferences, we can touch on additional topics in the agenda, such as (i) using algorithmic statistics to distinguish epistemic and aleatoric uncertainty (ii) the formalization of instrumental goals (iii) reinforcement learning with imperceptible rewards.

Keywords

• Infrabayesianism