Event Recap: Framing Made in Africa AI Approaches to MERL

On May 15th, the AI+Africa Working Group at the Natural Language Processing Community of Practice (NLP-CoP) hosted its first virtual gathering of 2025, bringing together over 45 participants who are members of the working group working in different development and humanitarian sectors. Following our inaugural meeting in May 2024 and a successful fundraising effort that secured dedicated personnel, this session aimed to reignite momentum and strengthen collaboration.

Our speakers were:

1. Nana Mgbechikwere Nwachukwu, AI Ethics and Governance Expert, who led discussion on ethics and governance;
2. Israel Olatunji Tijani, Data Scientist and Chatve Founder, discussing guidance on LLM development in Africa.

Given the rapid evolution of the AI landscape, the gathering focused on understanding Africa’s current position regarding AI use and development in evaluation. We also sought insights from members on how the AI+Africa Working Group and the NLP-CoP could better support their needs. Key topics discussed included identifying knowledge gaps and emerging skills in MERL+AI, visions for ‘Made in Africa AI & Evaluation,’ preferred engagement platforms, and ways to build sustainable collaborations.

Where are we with the use and uptake of AI in Africa?

The speakers led conversations emphasizing the fragmented regulatory environment across Africa. Despite increasing policy attention, such as the April 2024 Kigali Declaration, harmonized and enforceable data protection laws remain scarce. The under-ratification and limited enforcement of the Malabo Convention significantly hinder cohesive AI governance. There are contrasts with European frameworks like GDPR and the EU AI Act, highlighting Africa’s vulnerabilities regarding data sovereignty and accountability.The conversations urged stakeholders to stop duplicating external frameworks and instead design governance systems that respond directly to Africa’s unique continental context.

Enforceable legal frameworks beyond broad strategies are lacking, meaning AI investment risks becoming exploitative or extractive. Civil society was urged to assert greater influence in shaping accountability systems, especially given the projected $60 billion investment in AI on the continent, which currently lacks transparency regarding who will be the custodian of this process to see it through to fruition.

Patent and intellectual property protections were highlighted as crucial yet lacking. Weak legal frameworks around patents and copyrights expose African innovations to theft, under-referencing, and global co-option without due credit. This invisibility of African innovation reinforces technocolonialism and epistemic bias, which is detrimental to Africa-led innovation. 

Insufficient technical infrastructure and compromised digital sovereignty due to dependence on foreign cloud providers (Google Cloud, AWS) severely limit African AI growth. South Africa, uniquely possessing local data centers, contrasts sharply with broader continental inadequacies, which need to be taken into account when we are discussing the future of Africa digital public infrastructure. Poor broadband connectivity further exacerbates these challenges, especially during AI user testing and pilot deployments.

Environmental sustainability emerged as a pressing concern, with speakers illustrating this through the resource-intensive nature of AI infrastructures like data centers. They pointed to Google’s use of 22.7 billion liters of water for cooling in 2022 alone. They emphasized the critical need to understand and measure trade-offs in ways that respond to Africa’s specific climate realities and impacts. Speakers urged investment in infrastructure that avoids extractive and harmful practices and instead centers on climate and environmental justice.

Bias in language models and tools, primarily trained on English and Euro-American datasets, adversely impacts data quality and contextual representation in Africa. An illustrative example was the Yoruba word “dupe,” meaning gratitude, mistakenly categorized as “dupe”  by Meta’s NLP tools. Speakers emphasized integrating ethics from ideation to design stages, reinforcing the foundations for a Made in Africa AI approach.

Speakers emphasized the urgent need to localize evaluation benchmarks in AI and MERL processes across Africa. They noted that most organizations still rely on Western-centric standards and data collection frameworks, despite these often being misaligned with African social and cultural contexts. One participant illustrated this disconnect through the example of the term “primary caregiver”, which in many African communities may refer not just to parents but also to extended kin like grandmothers or aunts. However, many evaluation tools enforce narrow, binary classifications rooted in Western family structures, which exclude or misrepresent caregiving realities in Africa. As one participant shared, “The templates we’re using to collect data aren’t African-centric… whoever designs the tool carries the power.” This misalignment affects both data quality and program design, potentially skewing evaluations and undermining meaningful participation for Made in Africa evaluations. 

MERL & AI Skills Needs

Participants highlighted the need for practical, skill-focused exchanges across countries, including:

• Training in AI governance and ethics: There is a need to equip MERL practitioners with the knowledge to question, influence, and co-create ethical AI tools from the outset. As emphasized during the discussion, ethical considerations should guide AI development before deployment, ensuring alignment with African sociocultural realities and avoiding harm.
• Legal literacy regarding IP and data laws: Several speakers stressed the lack of awareness around intellectual property laws and data ownership. Without this legal literacy, African innovators risk exploitation, with ideas and datasets extracted and repackaged by external actors.
• Contextual data collection tools: There is a gap in data tools that reflect African linguistic, cultural, and social norms. Many data templates continue to replicate foreign models, leading to poor-quality data and exclusion.
• MERL frameworks integrating AI while respecting African value systems: Participants advocated for evaluation approaches that go beyond imported metrics and reflect indigenous values, community worldviews, and developmental aspirations. This includes co-creating indicators with local actors and ensuring that AI tools serve collective well-being—not just technical efficiency.

Opportunities

Amid these challenges, several promising opportunities emerged:

1. Inclusive AI design: The example highlighted was an AI-powered robotic sign language interpreter potentially serving Africa’s 14 million individuals with hearing loss, illustrating how locally designed tools can bridge structural inequalities.
2. Drone technology for MERL: Drones offer improved data collection in remote or fragile areas, though ethical deployment is essential.
3. Local language NLP tools: These can significantly enhance information access, and education, and preserve indigenous languages and traditions.
4. Data co-creation models: Shifting from extractive to participatory, community-based data collection ensures ethical standards and contextual relevance.

Working Group Reflections

Participants offered reflective insights aimed at strategic action, which we’ve listed below:

African leadership. Participants in the session posited the need for African nations to move beyond seeking external validation and instead “bargain for our own fair share” by strengthening infrastructure, improving digital literacy, and holding governments accountable through national AI strategies that align with AU Continental guidelines. The working group celebrated Kenya’s proven leadership in innovations like M-PESA and data-centred organizations such as Ushahidi as evidence that Africa can “pave the way” rather than waiting for inclusion in Global North boardrooms like OpenAI’s.

Made in Africa AI. The working group envisioned AI that directly serves African economies and communities. Participants emphasized that “made in Africa AI” should build products that improve grassroots livelihoods and generate revenue for both individuals and nations, transforming beyond buzzwords to achieve measurable impact on people’s lives.

African evaluation templates. Deeper epistemological reflections critiqued the dominance of Global North evaluation templates, arguing they shape data collection and narratives about African people inaccurately. The working group examined the epistemological foundations of current evaluation practices. Thoe noted how Global North templates—featuring Western constructs like ZIP codes and binary “Mr./Mrs.” labels—fail to capture African cultural realities. The working group championed decolonized data collection tools that authentically reflect local norms and values.

Structured peer learning among MERL practitioners emerged as a key strategy for advancing African-driven AI evaluation. Practitioners proposed data sovereignty as a strategic framework, emphasizing local evidence generation through co-creation and real-time intervention testing within African ecologies. The group identified structured peer learning through regional associations like the South African M&E Association (SAMEA) and the African Evaluation Association (AfrEA) as essential mechanisms for influencing continental AI policy and practice in MERL.

Safe and ethical approaches. Finally, participants raised important concerns about the psychological well-being of African data annotators working on harmful content for Big Tech companies. Data annotators manually review, categorize, and label vast amounts of text, images, and other content to train AI models. For example, they might tag thousands of social media posts as “hate speech” or “safe content,” or label images to help AI systems recognize objects and people. This work is foundational to AI development because machines learn from these human-created labels to make decisions about new content they encounter. The group advocated for comprehensive psychological support and trauma-informed employment practices, particularly as global tech companies increasingly partner with African workers for content moderation and data labeling in places like Kenya. This highlights the need for ethical labor standards in AI development partnerships.

What is next for the AI in Africa Evaluation Working Group?

Moving forward, the Working Group emphasized three core priorities: 

• advancing AI integration in MERL practices and investing in capacity building for AI integration
• fostering a distinctly Made in Africa Evaluation approach
• ensuring all AI developments adhere strictly to principles of Ethical and Safe by Design.

As a next step, Working Group members are invited to participate in key informant interviews, allowing deeper exploration of these themes and supporting further strategic action. You can reach out to vari@merltech.org to express your interest in participating in a key informant interview as a MERL in Africa Practitioner. 

We’ll be using these insights that we gain to help us develop a Made in Africa AI Landscaping Report, plan future events, and develop resources and an overall work plan for the AI in Africa Evaluation Working Group. We’ll invite you to a findings report validation meeting in the last week of June, so mark your calendars for June 24!  In the meantime, we hope you’ll find this detailed recap useful. We would also love to have suggestions for speakers and to spotlight your work, so if you are interested, please submit your ideas here. 

* If you would like to suggest events themes, speakers, and any ideas related to this working group, you can do so here.

* If you’re not already a member of the NLP-CoP, join here. You’ll also be able to sign up to one or more of our many Working Groups, including Ethics & Governance, SBC, and Humanitarian AI.