Sign in Subscribe
internet governance

International AI Safety Report: A Conversation with Shalaleh Rismani of Mila - Quebec AI Institute Institute

Inside the thinking behind the International AI Safety Report’s newest update on AI capabilities and risks.

International AI Safety Report: A Conversation with Shalaleh Rismani of Mila - Quebec AI Institute Institute
Photo by Claudio Schwarz / Unsplash

By Audrey Hingle in conversation with Shalaleh Rismani

The International AI Safety Report brings together research from experts around the world to provide a shared evidence base on the capabilities and risks of advanced AI systems. IX’s Mallory Knodel saw the main report presented at the United Nations General Assembly earlier this year, where it was introduced as part of an effort to inform global cooperation on AI governance.

To better understand the thinking behind the report and its recent update, I spoke with Shalaleh Rismani of Mila - Quebec AI Institute, one of the authors of the recent First Key Update. The update focuses on rapid advances in AI reasoning capabilities and examines how those developments intersect with emerging risks, including cybersecurity, biological threats, and impacts on labor markets. You can read both the report and the update at internationalaisafetyreport.org

Why this report, and why now? What gap did the team hope to fill in the global AI safety conversation?

This is the second year the safety report has been produced as a collaborative project. The main report’s scope was set early by the lead writers and panelists, with input from experts around the world. The goal was to synthesize evidence on the most advanced AI systems, including technologies already being rolled out and others still in development, in a way that would be useful for policymakers.

As the field evolved, the team realized that one annual report was not enough to keep up with the pace of change. This year, the leadership decided to produce two interim updates in addition to the main report. The first, released in October, focused heavily on capabilities, particularly what researchers refer to as “reasoning capabilities.” These include systems that can generate multiple possible answers or ask clarifying questions before responding. The second update, coming at the end of November, will continue tracking those advances, while the next full report will be published in February.

The report cites thousands of studies. How did the team ensure that this huge body of research remains usable for policymakers and practitioners?

The main goal is to bring in as much evidence from the academic literature as possible and make it accessible to policymakers and the public. Each section is led by researchers embedded in the literature, and multiple rounds of revisions happen with expert reviewers.

Every citation goes through a vetting process to confirm that it comes from credible academic sources. Because AI research moves so fast, much of the work is pre-published, which makes it harder to assess. Still, the idea is to present the full range of research and show both where strong evidence exists and where gaps remain.

Publishing is one thing, but ensuring impact is another. How does the team think about getting the report in front of key audiences?

The dissemination strategy is a collaborative effort between the Chair, the writing team and the secretariat. The team participates in many briefings with governments and policymakers around the world. For example, we engaged directly with policymakers on the findings of the first key update, including from the EU, India, UK, Canada, Singapore, UAE, Australia, Japan, Kenya and others. Because panelists, senior advisers, and reviewers come from different countries, there is already strong buy-in. Civil society, academia, and major technology companies are also involved in the process, which helps expand the report’s reach.

How did the team integrate human rights considerations into what is otherwise a very technical safety framework?

Human rights are not presented as a standalone section, but they are integrated throughout the report. One way is by identifying where evidence exists and where it does not, which highlights gaps relevant to fairness, privacy, and equity. Many evaluations measure performance on benchmarks but not real-world outcomes. Pointing out those gaps helps guide future human rights work by showing where contextual studies are needed.

Some of the risks discussed in this update also touch directly on human rights. For example, the growing adoption of AI companionship technologies raises concerns about loneliness and emotional well-being. The report also notes early evidence of labor market impacts, particularly in software engineering, although broader economic effects are still unclear.

The report came out of a large international process. What did that collaboration reveal about where consensus exists and where it still breaks down when it comes to defining and governing AI safety?

There is broad agreement that AI systems are improving on certain benchmarks, but less consensus on whether those benchmarks accurately measure complex abilities like reasoning. Some experts question whether the current evaluation frameworks are valid for assessing reasoning at all.

There is also consensus that potential risks should be monitored proactively rather than ignored, though there is debate about which risks are most pressing. Monitoring and controllability risks, for instance, are still contested. Some lab studies suggest models underperform when they know they are being evaluated, while others do not show this effect. In contrast, there is stronger agreement around risks such as AI companionship, labor market disruption, and cyber offense and defense.

The report brings together such a wide range of evidence and perspectives. How do you think about assessing risk and avoiding overhyping progress?

The report does not use a specific framework to assess risk. There are frameworks being proposed for evaluating AI systems, and we report on developments in those frameworks rather than applying one ourselves.

We also recognize the risk of overhyping AI progress, especially right now. To address this, we try to look for real-world evidence of both improvements and shortcomings. The review processes and involvement of stakeholders are other ways this can be managed and help keep the report balanced.

If you had to highlight one or two takeaways that you hope will shape AI policy or practice in 2026, what would they be?

There is a significant gap in evaluating real-world impacts. Policymakers need a clearer understanding of how AI systems affect work, research, and society, not just benchmark scores. Creating infrastructure to support independent evaluations and audits will be key, whether through third-party organizations or public feedback mechanisms.

The second update, coming later this year, will focus on risk management practices and the solutions being proposed to address them. The goal is to show that progress is happening while recognizing that there is still much more work to do.

IX at MozFest

We’re back from our recent session at MozFest and buzzing with excitement from all of the ideas and connections we made. The room was packed for our session, Encryption and Feminism: Reimagining Child Safety Without Surveillance, and the conversation went far beyond the usual encryption talking points. Participants pushed into the real tensions between safety and care, shared lived experiences of how surveillance harms survivors and marginalised communities, and offered concrete ideas for what genuinely feminist approaches to child safety could look like.

We’re now working through the feedback forms and pulling those insights into a draft set of feminist principles for encryption. We’re also exploring an online rerun so more people can join the discussion and contribute, since not everyone interested could make it to MozFest and many at MozFest who wanted to attend couldn’t fit everyone into the room. So stay tuned!

Support the Internet Exchange

If you find our emails useful, consider becoming a paid subscriber! You'll get access to our members-only Signal community where we share ideas, discuss upcoming topics, and exchange links. Paid subscribers can also leave comments on posts and enjoy a warm, fuzzy feeling.

Not ready for a long-term commitment? You can always leave us a tip.

Become A Paid Subscriber

From the Group Chat 👥 💬

This week in our Signal community, we got talking about:

IX contributor Heather Burns blew up Hacker News and even got a shout-out in the Financial Times for her viral blog post “Time to start de-Appling,” which warns UK users to migrate their data off iCloud after Apple announced it will disable Advanced Data Protection under the Investigatory Powers Act. Heather argues the move exposes how post-Brexit tech policy is eroding privacy rights. The post comes hot on the heels of our MozFest session on encryption and feminism, where we explored similar themes: governments invoke “child safety” to justify weakening encryption, sometimes at the expense of the very people it protects.

Read The Post

Open Social Web

Internet Governance

Digital Rights

Technology for Society

Privacy and Security

Upcoming Events

Careers and Funding Opportunities

United States

Global

Opportunities to Get Involved

What did we miss? Please send us a reply or write to editor@exchangepoint.tech.

💡
Want to see some of our week's links in advance? Follow us on Mastodon, Bluesky or LinkedIn, and don't forget to forward and share!

Read next

Comments ()