April 22, 2026

Safety by Design: When AI Finds the Cracks, Who Falls Through Them?

Guest post by Sara Chamberlain, with contributions from Sofie Meyer and Isabelle Amazon-Brown.

If you design, fund, procure, or evaluate digital development programs, AI risks are outpacing our sector’s response. This blog explains what’s at stake, and what we can do about it.

AI’s impact is global. Its governance is not. The world’s most powerful models are being designed by companies with little accountability to the governments and communities most vulnerable to the risks they create.

Anthropic’s recent announcement that its new frontier model, Claude Mythos, had identified thousands of vulnerabilities across every major operating system and web browser makes clear just how significant those risks are.

At risk: Digital Public Infrastructure

Linux underpins government payment systems, digital identity platforms, health databases, and mobile money services across LMICs. These are systems that hold sensitive citizen data. Exploiting them does not just disrupt services. It exposes the personal, financial, and health records of millions of people who have little means of recourse.

The Linux kernel, the open-source foundation of much of this infrastructure, was among the systems Mythos examined. Researchers gave Mythos 100 known Linux weaknesses, and it identified which flaws were the most dangerous, then wrote working attack code for more than 20 of them, entirely on its own, after a single instruction. For every vulnerability that goes unpatched, the window of opportunity for attackers grows. Fixing them requires technical expertise and resources that many LMIC governments are already stretched to provide.

This risk is emerging while governments in LMICs are already being pushed to trade data sovereignty for funding: Kenya’s $2.5 billion health data deal with the US and the wave of similar bilateral agreements now sweeping across Africa are examples. Kenya’s courts temporarily halted the deal over data privacy concerns, but the pressure on LMIC governments to accept unfavourable terms is only growing.

The accountability gap

Stanford’s 2026 AI Index Report found that documented AI incidents rose to 362 in 2025, up from 233 the previous year, a surge of over 55%, but at the same time, model transparency has declined. The Foundation Model Transparency Index, developed by researchers at Stanford, MIT, Princeton, and UC Berkeley, scores how openly AI companies share information about their most powerful models. After improving from 37 to 58 out of 100 in 2024, the average score dropped back to 40 in 2025, meaning developers are sharing less, not more, as their models become significantly more powerful. What they are not disclosing, including what data their models were trained on and what real-world impact they are having, is precisely what governments, researchers, and affected communities need to independently assess whether these systems are safe.

The Winter 2025 AI Safety Index found that no major AI company scored above a D on existential safety planning, meaning none have adequate plans for maintaining human control if their systems begin behaving in dangerous and unintended ways. We are being asked to trust the most powerful AI systems ever built largely on faith.

And yet the global conversation about who is most exposed to this accelerating threat landscape has barely begun.

The equity gap

Some of the most vulnerable people are low-income women in LMICs, and the barriers to their protection are not just technical. They are deeply social, economic, and in many cases, built into the design of the tools themselves (for example, authentication systems that rely on SMS one-time passwords that illiterate or non-numerate women simply cannot read).

GSMA research identifies safety and security concerns as among the top three barriers for male and female mobile internet users in all 15 survey countries except Ethiopia, and for women in Pakistan. These fears do not operate in isolation. Male gatekeepers limit women’s technology use based on disproportionate concerns about women’s vulnerability to fraud online. The result is a double bind: women are afraid of fraud, men use that fear as a justification for control, and the underlying risk environment goes unaddressed.

Research across India, Cambodia and Kenya find that only a minority of women have ever changed a password on a mobile device or online account. In the Indian state of Bihar, Evidence for Digital Transformation found that just 5% of women have ever changed a social media password. In Kenya, She Mobilzes found that only 21% of women in its digital skills programme had ever changed an online account password. Even among garment factory workers in Cambodia, who almost all own smartphones, She Mobilizes found that 55% of women learners had never used a phone lock. This is not apathy. It is a predictable consequence of exclusion from the design decisions shaping every interface they encounter

When the “solution” assumes literacy

Two-factor authentication via SMS is the most widely recommended protection against account fraud. For a significant proportion of women in LMICs, it is out of reach. According to data released by the UNESCO Institute for Statistics (UIS) in September 2025, approximately 739 million adults (aged 15+) globally lacked basic literacy skills in 2024–2025, with women representing nearly two-thirds of this population

In India, data from the National Family Health Survey 5 shows illiteracy and low literacy rates among women aged 15 to 49 that are hard to ignore: 55.1% in Bihar, 52% in Madhya Pradesh, 48% in Rajasthan, 46% in Uttar Pradesh. Reading a one-time password sent by text message is not a minor hurdle for a woman who cannot read. It is an absolute barrier. The continued reliance on this approach places women at higher risk of fraud, data breaches, and accidental disclosure.

Phone sharing amplifies vulnerability

Across LMICs, large numbers of women share phones with husbands, in-laws, and children. Every shared device is a compromised security boundary. Every shared account is an exposed identity. And yet digital platforms continue to be designed for a single owner with uninterrupted access and the ability to read.

Safety by Design: what our sector needs to be asking of technology developers

Design authentication for low-literacy contexts. Voice-based and biometric verification are not premium features; they are equity features. Relying on ‘OTPs’ (SMS codes) as the only option potentially excludes hundreds of millions of people.
Design for shared device realities. Platforms that assume a single owner build risk into the experience for women who share phones. That is a design choice, not a technical inevitability.
Be transparent. A transparency score of 40 out of 100 for the world’s most powerful AI systems, at a moment when those systems are finding exploitable weaknesses in the very infrastructure that stores citizen data across LMICs, is not a technicality. It is a governance failure. Full disclosure of how these models are built, what data they were trained on, and what real-world harms they are causing is a minimum requirement, not a courtesy.
Take existential safety planning seriously. No major AI company currently scores above a D on existential safety planning, meaning none have adequate strategies for maintaining human control if their systems begin behaving in dangerous and unintended ways. That is not acceptable.

Building safety into the AI tools we design is necessary but not sufficient. As a community, we must also advocate for:

Digital safety and security training for vulnerable populations. Low-literate women in particular need sustained, accessible, peer-led training on how to protect themselves from fraud and scams. This cannot be one-off, text-heavy, or pitched at the wrong level.
Investment in governments’ capacity to protect their citizens’ data. The technology teams responsible for managing and securing digital public infrastructure in LMICs need dedicated investment to understand, respond to, and recover from AI-accelerated security threats. These systems hold the financial, identity, and health records of millions of people. The governments responsible for them are on the front line of a risk landscape they did not create and are not yet equipped to navigate.
Advocating for regulation of AI giants. Voluntary transparency scores and self-reported safety planning are not enough. Governments, civil society, and the development community need to actively push for binding regulation that requires AI companies to disclose how their models are built, demonstrate that safety standards are met, and be held accountable when they cause harm. We have more standing in this conversation than we realise. Using it is not overreaching. It is overdue.

The question Safety by Design asks is not only can we find the cracks faster? It is: who designed a world where the most vulnerable people fall through them first, and what would it look like to design it differently?

Designing differently is exactly what we are trying to do together. The MERL Tech Initiative’s Design & UX of AI Learning Group is a space for development and humanitarian professionals to explore how AI is reshaping design and user experience, and to ask hard questions about what it means to design safely, equitably, and for the communities most at risk. If the issues raised in this blog resonate with your work, we’d love you to join us – out next event will be on designin safe AI tools. Sign up via the NLP CoP pageor reach out to isabelle@merltech.org to volunteer as a speaker.