Reasoning about uncertainty is one of the crucial components informing human intelligence and our very survival — it can encourage us to take risks that have a higher expected return but also discourage us from impulsively doing things that may lead to catastrophic consequences. Contemporary Bayes-optimal AI agents however are generally risk- and ambiguity-neutral, lacking this natural human capacity for advanced reasoning with regard to uncertainty.
In the new paper Beyond Bayes-Optimality: Meta-Learning What You Know You Don’t Know, a research team from DeepMind and Stanford University employs modified meta-training algorithms to build agents with risk- and ambiguity-sensitivity, and empirically demonstrates the validity of such agents.
The team first clarifies the distinction between Bayesian-optimal, risk-sensitive, and ambiguity-sensitive agents. Simple put, both risk and ambiguity belong to uncertainty: risk applies to scenarios where the output of an event is uncertain but can be roughly calculated using probability functions; while in ambiguity, the probability is unknown or cannot be reliably determined. Bayesian-optimal agents assign certainty-equivalents that are equal to the expected payoff, then choose the optimal action. Such agents are both risk-neutral (insensitive to the distribution over returns except for the target value) and ambiguity-neutral (acting as if the uncertainty were known).
In their bid to create risk-sensitive agents, the team modifies the meta-training protocol by tweaking the distribution of observations to make these observations sensitive to the valuations of the agent. They leverage an ensemble of agents and a meta-policy to build a mechanism able to detect and use novelty. As such, the resulting agent can evolve to become uncertainty-seeking and risk-averting based on its experiences.
In their empirical study, the team applied their proposed meta-training algorithms to agents in various decision-making experiments, with the results confirming that the agents can learn both risk- and ambiguity-sensitivity.
Overall, this work demonstrates the possibility of building risk- and ambiguity-sensitive agents via the modification of meta-training algorithms. The team hopes their contributions can serve as the starting point in the development of data-dependent methods for the study and application of uncertainty-sensitivity in humans and machines.
The paper Beyond Bayes-Optimality: Meta-Learning What You Know You Don’t Know is on arXiv.
Author: Hecate He | Editor: Michael Sarazen, Chain Zhang
We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.