Publications | Shahar Avin

Computing Power and the Governance of Artificial Intelligence

Analysis of compute as a lever for AI governance.

Girish Sastry, Lennart Heim, Haydn Belfield, Markus Anderljung, Miles Brundage, Julian Hazell, Cullen O'Keefe, Gillian K. Hadfield, Richard Ngo, Konstantin Pilz, George Gor, Emma Bluemke, Sarah Shoker, Janet Egan, Robert F. Trager, Shahar Avin, Adrian Weller, Yoshua Bengio, Diane Coyle

PDF

Lessons from COVID-19 for GCR governance: a research agenda

Research agenda for studying COVID-19, and the COVID-19 response, through a Global Catastrophic Risk lens.

Jochem Rietveld, Tom Hobson, Shahar Avin, Lalitha Sundaram, Lara Mani

PDF

Frontier AI regulation: Managing emerging risks to public safety

The case for regulating frontier AI models.

Markus Anderljung, Joslyn Barnhart, Jade Leung, Anton Korinek, Cullen O'Keefe, Jess Whittlestone, Shahar Avin, Miles Brundage, Justin Bullock, Duncan Cass-Beggs, Ben Chang, Tantum Collins, Tim Fist, Gillian Hadfield, Alan Hayes, Lewis Ho, Sara Hooker, Eric Horvitz, Noam Kolt, Jonas Schuett, Yonadav Shavit, Divya Siddarth, Robert Trager, Kevin Wolf

PDF

Model evaluation for extreme risks

The need to evaluate advanced AI for dangerous capabilities.

Toby Shevlane, Sebastian Farquhar, Ben Garfinkel, Mary Phuong, Jess Whittlestone, Jade Leung, Daniel Kokotajlo, Nahema Marchal, Markus Anderljung, Noam Kolt, Lewis Ho, Divya Siddarth, Shahar Avin, Will Hawkins, Been Kim, Iason Gabriel, Vijay Bolina, Jack Clark, Yoshua Bengio, Paul Christiano, Allan Dafoe

PDF

Strengthening Resilience to AI Risk

UK policy guide to AI risk and governance.

Ardi Janjeva, Nikhil Mulani, Rosamund Powell, Jess Whittlestone, Shahar Avin

PDF

Frontier AI regulation blueprint

A high level blueprint for domestic regulation of civilian advanced AI models. Created with the help of Cotton Design

Shahar Avin

Interactive version Read-only version Blog post

Rewards, risks and responsible deployment of artificial intelligence in water systems

Risk assessment of AI in water systems.

Catherine E. Richards, Asaf Tzachor, Shahar Avin, Richard Fenner

PDF

Responsible artificial intelligence in agriculture requires systemic understanding of risks and externalities

Risk assessment of AI in agriculture.

Asaf Tzachor, Medha Devare, Brian King, Shahar Avin, Seán Ó hÉigeartaigh

PDF

Filling gaps in trustworthy development of AI

Incident sharing, auditing, and other concrete mechanisms could help verify the trustworthiness of actors

Shahar Avin, Haydn Belfield, Miles Brundage, Gretchen Krueger, Jasmine Wang, Adrian Weller, Markus Anderljung, Igor Krawczuk, David Krueger, Jonathan Lebensbold, Tegan Maharaj, Noa Zilberman

Preprint PDF DOI Science Gizmodo

An agent-based model clarifies the importance of functional and developmental integration in shaping brain evolution

Computer simulation explores brain size evolution.

Shahar Avin, Adrian Currie, Stephen H Montgomery

Preprint PDF Code

Tackling threats to informed decision-making in democratic societies

Analysing technological threats to epistemic security.

Elizabeth Seger, Shahar Avin, Gavin Pearson, Mark Briers, Seán Ó Heigeartaigh, Helena Bacon

PDF

Towards Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims

Ten mechanisms for supporting the verification of claims made by developers of AI systems.

Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, Tegan Maharaj, Pang Wei Koh, Sara Hooker, Jade Leung, Andrew Trask, Emma Bluemke, Jonathan Lebensbold, Cullen O'Keefe, Mark Koren, Théo Ryffel, JB Rubinovitz, Tamay Besiroglu, Federica Carugati, Jack Clark, Peter Eckersley, Sarah de Haas, Maritza Johnson, Ben Laurie, Alex Ingerman, Igor Krawczuk, Amanda Askell, Rosario Cammarota, Andrew Lohn, David Krueger, Charlotte Stix, Peter Henderson, Logan Graham, Carina Prunkl, Bianca Martin, Elizabeth Seger, Noa Zilberman, Seán Ó hÉigeartaigh, Frens Kroeger, Girish Sastry, Rebecca Kagan, Adrian Weller, Brian Tse, Elizabeth Barnes, Allan Dafoe, Paul Scharre, Ariel Herbert-Voss, Martijn Rasser, Shagun Sodhani, Carrick Flynn, Thomas Krendl Gilbert, Lisa Dyer, Saif Khan, Yoshua Bengio, Markus Anderljung

PDF VentureBeat Financial Times

Exploring AI Futures Through Role Play

Exploration of AI futures and impacts through participatory role-play simulation.

Shahar Avin, Ross Gruetzemacher, James Fox

PDF Project DOI

Autonomy and machine learning at the interface of nuclear weapons, computers and people

Describes new threats posed by the introduction of ML at the interfaces between people and nuclear weapons systems (and related systems), and proposes policy responses.

Shahar Avin, SM Amadae

PDF DOI SIPRI complete volume

Method Pluralism, Method Mismatch & Method Bias

Modelling different approaches to evidence gathering.

Adrian Currie, Shahar Avin

Preprint Code

Surveying Safety-relevant AI Characteristics

Survey of known and potential safety-relevant AI characteristics, covering internal characteristics, system-environment effects, and training.

Jose Hernandez-Orallo, Fernando Martínez-Plumed, Shahar Avin, Seán Ó hÉigeartaigh

PDF

Exploring artificial intelligence futures

Reviews different methods to explore AI futures, including fiction, single-discipline studies, several multidisciplinary approches, and interactive methods.

Shahar Avin

PDF

Mavericks and Lotteries

Summarises academic arguments for funding science by lottery, and compares these to real implementations of the policy.

Shahar Avin

PDF

Beyond Brain Size: Uncovering the Neural Correlates of Behavioral and Cognitive Specialization

Limits of brain size measures in the study of brain-behavior across species, and suggestions for alternative research approaches.

Corina J Logan, Shahar Avin, Neeltje Boogert, Andrew Buskell, Fiona R Cross, Adrian Currie, Sarah Jelbert, Dieter Lukas, Rafael Mares, Ana F Navarrete, Shuichi Shigeno, Stephen H Montgomery

Preprint PDF

Policy Considerations for Random Allocation of Research Funds

Expanding on the policy implications of the model results in my PhD thesis.

Shahar Avin

PDF Project

Classifying global catastrophic risks

Classifying GCRs based on critical system breached, global spread mechanism, and human prevention and mitigation failure.

Shahar Avin, Bonnie C Wintle, Julius Weitzdörfer, Seán S Ó hÉigeartaigh, William J Sutherland, Martin J Rees

PDF Slides Video

The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation

Risks and responses to near-term malicious use of AI/ML.

Miles Brundage, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Bobby Filar, Hyrum Anderson, Heather Roff, Gregory C Allen, Jacob Steinhardt, Carrick Flynn, Seán Ó hÉigeartaigh, Simon Beard, Haydn Belfield, Sebastian Farquhar, Clare Lyle, Rebecca Crootof, Owain Evans, Michael Page, Joanna Bryson, Roman Yampolskiy, Dario Amodei

Preprint PDF Video BBC New York Times Wired Opinion Wired The Verge CyberLaw podcast SecurityNow World Economic Forum People's Daily כלכליסט Deutsche Welle L'actualité

Centralized funding and epistemic exploration

Extension of the model developed in my PhD thesis.

Shahar Avin

Preprint PDF Code Project

Science funding is a gamble so let’s give out money by lottery

Popular version of the argument made in my PhD thesis.

Shahar Avin

Project Aeon

Funding science by lottery

Early quick overview of the argument made in my PhD thesis.

Shahar Avin

Preprint PDF Project

Breaking the grant cycle: On the rational allocation of public resources to scientific research projects

Agent-based simulations of hypothetical research communities show random allocation can outperform selection by potential merit under certain conditions.

Shahar Avin

PDF Project Poster

Why We Still Need Grant Peer Review

A comment on “From funding agencies to scientific agency” by Bollen et al.

Shahar Avin

Preprint PDF