Research

We conduct long-term future research with a focus on investigating AI race dynamics. On the one hand, we use game theory to analyze games similar to the game presented in the publication Racing to the Precipice; on the other hand, we use computational modeling to analyze models which are intended to be more representative of real-world AI development dynamics. To achieve the latter, we formalized and implemented AI race models that combine best practices from game theory, economics, technological races, and AI safety literatures. While we love sharing our insights, we always make sure that they aren't considered information hazards. Therefore, some of our research isn't publicly available.


Numerical optimal solution

In the paper Racing to the Precipice Racing to the Precipice, Armstrong, Bostrom, and Shulman presented a static AI race game. We built a probabilistic version of their game and identified the Nash Equilibrium strategies for the extended model using a computational approach. As part of this analysis, we discuss how uncertainty and differing marginal returns from safety investments jointly influence the risk of disaster.

Windfall Clause examination

The Future of Humanity Institute has proposed a Windfall Clause policy as a possible way to decrease risks from AI races. We applied this policy to the model presented in Racing to the Precipice and examined how the policy changes the disaster risk of an AI race in different scenarios. We found that it is rational for race participants to join the Windfall Clause, and thereby reduce disaster risk, in a surprisingly large number of modeled AI races. This suggests that the Windfall Clause can be an effective tool for mitigating the worst dangers of AI competition.