Do you see what I see? Insights from an inclusive approach to AI ethics governance.

In the past 2 years, the development and humanitarian sectors have had to rapidly create strategies, guidance and policies on how AI can and should be used, safely and responsibly, in response to the emergence of Large Language Models (LLMs) and Generative AI. But discourse and governance on ethical AI continue to be dominated by white Westerners, with end-users of AI, and stakeholders in the Global South, unevenly consulted.

Last year, The MERL Tech Initiative (MTI)  was commissioned to develop guidance on ethical AI for Girl Effect, an INGO which connects girls and young women with the resources and support they need to meet their full potential, including via digital tools like AI-powered chatbots. As part of this, we designed and carried out light-touch research to uncover whether the guidance we produced – based on current best-practice in ethical AI – was relevant and usable to various stakeholders. Although this guidance was intended for internal use (note – we’ve since agreed to share it publicly – watch this space!), we also wanted to find out what the girls and young women using Girl Effect’s digital tools knew about AI, and whether the ethical issues our sector is prioritising are the ones they care about the most. 

We discovered encouraging common-ground across staff members, vendors, and end-users in Africa, India and Europe, particularly on the issues of data privacy, reliability, and cultural relevance. But we also identified discrepancies across countries, not least in terms of girls’ ability to grapple with the complex reality of AI. Our research also suggested that as we roll out AI tools, end-users believe we also have a responsibility to deliver AI-literacy tools and content, to enable them to exercise more fully their own agency when it comes to navigating the safe use of AI.

*

AI-powered chatbots for sexual health

Since 2017, Girl Effect has been developing chatbots to provide girls and young women with private, reliable digital spaces in which to learn about sexual and reproductive health, relationships and mental health. Available on Instant Messaging apps like WhatsApp, Messenger and MoyaApp, these chatbots allow users to access vetted educational content, test their knowledge with quizzes, find appropriate real-world services, and get answers to their questions. 

Initially developed as ‘decision-tree’ style chatbots supported by human-in-the-loop functionality to ensure girls at risk of harm were identified and supported by trained experts, Girl Effect’s chatbots have evolved to incorporate AI in order to better signpost girls to the most relevant information. This originally included using a deterministic architecture (no AI), then a BERT classification model, before evolving to include using a generative model to more accurately carry out classification as well as answer girls’ questions.

Girl Effect’s Big Sis chatbot

As mapped out in their AI Vision paper, Girl Effect’s goal is ultimately to enable their chatbot users to get immediate, personalized, safe responses to their questions via GenAI. They have just completed a first live Beta test of their Gen-AI powered model, which has shown that users are:

• 11.24% more likely to recommend Big Sis
• 17.1% more likely to access key messaging
• 11.8% more likely to return
• 12.68% more likely to access service information
• Ask 200% more questions

As part of this process, they commissioned MTI to develop guidance on the design and development of ethical GenAI, rooted in Girl Effect’s existing work on Digital Safeguarding which MTI also supported. 

Helping teams navigate ethical AI

The ethical GenAI guidance, and accompanying implementation checklist, covers both top-level principles and more granular advice relating to the deployment of ethical GenAI for Social and Behaviour Change chatbots – much of which echoes digital best practice principles which have been in circulation (if not always in practice) for years. First and foremost of these is honestly questioning whether the technology is needed and appropriate – ‘opting out’ of AI remains a sensible approach in many situations.

For those, like Girl Effect, who have been working with AI for many years, the principles to carry forward include values like collaboration, inclusivity and participation, privacy by design, data protection, accessibility, and accountability. GenAI-specific considerations include principles of explainability and transparency (which have become crucial given the opaque nature of LLMs), and building in additional safeguards (given the tendency of LLMs to hallucinate). 

There are so many potential risks and unknowns involved in working with cutting-edge technology that it can be hard to know what to prioritise. Teams must grapple with the fact that ‘truly’ ethical GenAI is an impossibility at this moment in time, and this realisation can make them feel overwhelmed and powerless. One key step in the implementation checklist is, therefore, to be pragmatic, and focus on mitigating risks which have strategic importance to the organisation or programme.

For Girl Effect, this has meant focusing their initial efforts on making any GenAI answers highly:

• reliable (with the aid of RAG using expert-vetted content, as well as prompt-engineering)
• relevant (with contextual relevance, editorial values, tone and personality co-developed with users and embedded via prompt engineering)
• and safe (with guardrails implemented to avoid users receiving insensitive or toxic responses, and ongoing human in the loop functionality). 

Participatory approaches to AI governance

Research methodology

Once the first draft of our guidance was completed, we ran a consultation exercise with Girl Effect’s staff, AI vendors, and young constituents (end users) to understand whether it was clear, useful, and relevant – and whether it reflected and addressed their main concerns when it came to AI. This process revealed reassuring common ground, but also blind spots that readers may want to reflect on as part of their own exploration of AI, and their use of digital tools in general.

As part of our consultation, we spoke to 104 stakeholders with varying degrees of technical involvement and expertise around Girl Effect’s AI work, via surveys and Key Informant Interviews. Respondents were based in Africa, India, Europe, and the US. Our response rate was 74%, highlighting a generalised enthusiasm (or need) to engage with the topic of ethical AI. 

Respondents were asked about:

• their knowledge and understanding of AI and GenAI,
• their awareness of key ethical issues,
• which ethical issues mattered the most to them,
• their feedback on the guidance
• what message they would like to share with Girl Effect’s AI team.

Upskilling upstream for downstream impact

Reassuringly, the wider team’s biggest concerns, based on their current knowledge of ethical AI, matched those of the small team working on GenAI at Girl Effect and detailed above. Especially for those working in a non-technical capacity, our research showed that people had felt generally unequipped to tackle the ethical issues thrown up by AI, mostly because of a lack of knowledge, but also because tight timeframes and budgets can make it challenging to prioritise useful activities such as capacity building. Both those heavily involved in AI work and those working in ‘AI-adjacent’ roles, feel that working on ethical issues will get sidelined unless funders and programme designers include requisite time and money to address them. They pointed out that this needed attention will not be given unless:

• those in decision-making positions were also significantly upskilled when it came to the implications of creating ‘ethical-ish’ AI,
• consensus was articulated at a senior level (including with funders) on what ethical issues were both important, and feasible, to address

Until these changes happen, efforts to mitigate the risk of AI downstream will remain patchy and scatter-gun.

“To reduce uncertainty [on ethical AI], you need strong leadership, rapid decision-making and clear guidance, but that can be difficult to provide when there is so much uncertainty in the field itself.”

KII respondent

Improving diversity practices in governance and development of AI

Some respondents also pointed out that until the development of AI governance tools is led by those in Global Majority countries (which was not the case for this guidance), the field of AI ethics as a whole would remain biased. Similarly, they felt that stronger emphasis should be placed in the guidance on DEI hiring and procurement practices to ensure AI teams represent the full range of perspectives of those who use their products.

“A blind spot in this domain is I see no ethical frameworks around how we hire. Folks who contribute to artificial intelligence, are they representative of communities?”

KII respondent, South Africa

Watch out for the impact of AI development on staff

AI ethics is often focused on potential harms to end-users, but increasing coverage has been given to the impact of AI development on those building and evaluating LLMs – mostly those working on commercial models. Our consultation highlighted that this is relevant even at a smaller scale. The data curators we spoke to shared that data annotation and evaluation tasks, for example involving toxic responses or sensitive disclosures, could take a toll on their mental health. Further, data curators are often being asked to make complex decisions with profound ethical and political implications. In order to do their jobs effectively and safely, they need to be adequately supported and valued – and this needs to be emphasised more strongly in AI ethics guidance.

Where are African girls at with AI?

Girl Effect’s Youth Participation Approach means that country teams work closely with girls and young women from their target audience when developing programmes, brands, and digital tools. Tapping into this powerful opportunity to make our guidance itself more relevant and inclusive, we conducted a short 4 country survey. We received responses from 86 Youth Advisory Panel (YAP) members in South Africa, Tanzania, Kenya, and Ethiopia, who all have a strong involvement in Girl Effect’s work, including its activities with AI. 

The girls consulted came from a mix of backgrounds – respondents were from both urban and rural areas, including those with a degree and those who had not completed school, girls with children of their own, and who either owned or borrowed an internet-enabled phone from family members. Because of uneven sample sizes, our findings are anecdotal, but they suggest interesting avenues for further research and important glimpses into overlooked groups when developing AI governance.

Firstly, in terms of knowledge and understanding of AI, there were inevitable regional differences, with only 50% of Tanzanian respondents having heard of AI, and 62.5% having never used chatGPT – compared to 93% and 88.8% in Kenya. But even those in countries with higher general digital access and literacy, a generalised ‘awareness’ often masked a patchy understanding – for example, 9.3% of Kenya respondents who had heard of AI thought it stood for something completely different (not related to tech). 

I think it is a form of bringing girls together to get more information.”

Kenyan YAP answer to the question “What do you think AI means?”

AI literacy is needed

Similarly, even those respondents who had used a commercial tool such as ChatGPT were not always aware that they used Generative AI. Also interesting was the fact that, on average, 34% of respondents did not know whether Girl Effect was using AI in its digital tools or not, even in countries actively deploying AI-powered services. Both these findings feel significant because they confirm a phenomenon whereby girls and young women are engaging with powerful digital tools that carry complex risks, sometimes without being aware of it. Girls’ experience of AI is detached from any grounding in wider conversations about its safe use.

Perhaps because of this, the majority of respondents said they had no concerns about Girl Effect using Generative AI. Those who did were mostly worried this would impact Girl Effect’s own creativity and independent thinking – but also, the local relevance of its outputs. Despite this overarching positivity about AI, the majority of respondents told us that they believed it was Girl Effect’s job to teach girls how to use AI safely – with the exception of Tanzanian respondents, of which 43.8% said they didn’t know enough about AI to answer either way. 

By teaching girls about AI, Girl Effect can empower them to participate more fully in the digital world and use technology to their advantage. Additionally, (…) educating girls on the potential risks and benefits of AI can help them make informed decisions and avoid negative consequences.” 

– YAP member, South Africa

The ethical issues respondents were most aware of were fairly universal, with reliability and data risks coming out strongly, alongside inclusion and cultural relevance (although again, 46% of Tanzanian respondents said they had never heard of any of the issues) Interestingly, awareness of issues like labour rights was significantly higher in South Africa.  In terms of the issues girls cared most about, they mostly matched the awareness results, but inclusion and participation in the development of AI tools featured more heavily. In Ethiopia, respondents were especially concerned about the environmental harms caused by AI. 

*

Anecdotal insights like these suggest that whilst end-users often agree with the emphases placed on issues such as data privacy, reliability, and cultural relevance in their approach to ethical AI, organisations may want to dig deeper into the issues that their constituents care strongly about. Whilst there are some non-negotiable areas of focus designed to protect users from active harm, less explored, and yes, sometimes more complex areas such as AI supply-chains or environmental impact, may be worthy investments if we are to create tools which reflect the values that users care most about. Not only does this feel like the ‘right’ thing to do, but promoting the values underpinning these tools may also build trust and therefore, likelihood of continued adoption. 

Above all, through this research we learned how hungry end-users are for support in navigating their own independent journey with AI, and to get involved in conversations and decisions about ethical AI which affect them. When we asked them to share messages with the AI team, most echoed the sentiment of one respondent from South Africa, who requested that:

“Beyond educating girls on how to use AI safely, [INGOs] should consider how girls in various communities can contribute to AI development. (…) Girls should not just be consumers of knowledge but producers.“

When our sector took advantage of the mobile web and social media boom to start rolling out digital tools, we missed an opportunity, or dare we say, a responsibility, to build users’ agency through digital literacy initiatives to ensure that, whether they’re using our products or not, we’re doing what we can to help vulnerable communities to engage safely and productively with online spaces. Let’s not make that same mistake again.