Are we ‘listening’ or just collecting data?

The MERL Tech Initiative hosted a session on Tech-Enabled Community Listening (TECL) at the ICT4D Conference in Nairobi on May 20, 2025. We were joined by Africa’s Voices Foundation, Sattva, and I4DI, who shared lightning talks about their work. Below is a summary of the key insights and discussions from the session.

Mary Nzilani, Programmes Manager from Africa’s Voices Foundation (AVF), gave the first lightning talk. AVF designs systems for citizen-generated data, using radio, SMS, and IVR to reach marginalized voices and uses AI to analyze qualitative data at scale. They follow an 8-stage process from participatory design through citizen conversations through to analysis and actionable insights, with a key point of sharing findings and decisions back with community members.

Mary emphasized that high-quality, ethical tech-enabled community listening processes are not about tools. Rather, the goal is redistributing power through continuous feedback loops that allow citizens to shape what data is collected and the insights that emerge. She noted that open-ended, unconstrained participation often reveals hidden and sometimes uncomfortable realities, and that listening continuously can capture shifts, tensions, and undercurrents in what is happening in a community. She highlighted that “inclusion is not access; it is voice without filtering.”

Our second speaker, Lauren Ropp, from the Institute for Development Impact (ID4I), spoke about ID4I’s work, including impact evaluation, MEL systems, investment analytics and program design. Community listening can be integrated into these various areas. Lauren highlighted that tech can enable scale but can struggle with context. SMS, voice surveys, and chatbots offer breadth but miss tone and hesitation, group dynamics, and moments of silence. ID4I has found that TECL works best when digital tools are paired with face-to-face work – tech helps surface patterns, but we still need people to understand them in context. She emphasized that “scale is not a substitute for understanding.”

Prateek Jain from Sattva Consulting was our third speaker. Sattva’s work aims to bridge the gap between high-level policy and ground-level impact by integrating traditional research with next-generation tech-enabled methods, combining community insights, real-time pulse checks via various channels, and big data proxies to validate local stories and macro trends. A key lesson Sattva has learned is that “trust, transparency, and data feedback loops are non-negotiable in TECL.” Practical tips include building familiarity through pre-engagement with communities, being clear about the purpose of the listening exercise from the start, and always closing the loop so communities know what the results were.

How Community Listening has changed over the past 5 years

Lauren has seen important shifts driven by the COVID-19 pandemic, when “everything moved online, and it never went back.” Five years ago, MEL teams spent considerable time debating study design. Then COVID made digital the default, and budgets, timelines, and donor expectations never reset. A downside is that some organizations now view community listening as a budget line, procuring vendors who sell “listening as a service” rather than engaging directly. Community listening is treated as a procurement category rather than as a deep process for shifting power and building relationships.

Mary noted a related shift: a move in some cases from simply extracting data to “building living and responsive data ecosystems.” Digital advances allow organizations to move from one-off surveys with structured questions and delayed analysis toward continuous open listening through multiple channels and more rapid AI-assisted analysis. This offers potential for more authentic feedback and near-real-time insights for adaptive programming. The question remains, however, whether organizations have the capacity to make real-time decisions or are trapped in rigid structures that prevent rapid adaptation.

Prateek echoed this, noting that clients are seeking continuous, participatory, and scalable community engagement, with growing interest in participatory program design and a shift from compliance-driven to intent-led engagement. New technologies such as Voice AI are enabling scale, access, and more continuous engagement.

Key challenges raised for discussion

1)     What happens when what citizens say does not fit with donor priorities, log frames or results frameworks? 

Mary noted that when a space is opened for true listening and sharing with communities, what people say might not fit with what donors or organizations want or expect. People may move the conversation to a completely different topic. Organizations have to decide how they will make space for these conversations, even if they do not align with the original purpose or the expectations.

One participant suggested this might reflect on the design of the Theory of Change — were there missing assumptions, missing context, or missing risk? These “out of place” comments could signal whether a pathway is incomplete or a program needs to adapt. Mary agreed, yet felt that it’s still important to hold space for community members regardless, because they took time to share a part of themselves. She reminded the group that our institutional goals are not necessarily central, relevant, or even apparent to communities when they are sharing their ideas and opinions.

2)     Are we hearing more voices, or the same voices more efficiently? 

Lauren reminded us that in low and middle income countries (LMICs), women are 19% less likely than men to use mobile internet; rural women are 29% less likely than urban women. Mobile phone surveys consistently under represent women, rural, and older residents compared to in-person household surveys. Additionally, voice AI under performs on low-resource languages, non-standard dialects, and women’s voices. Each new layer of technology, she cautioned, inherits and compounds the biases of the last.

Participants noted that hybrid models can help — for example, by working with people who have greater access to support or serving as intermediaries for those with less. Others agreed it’s important to be intentional about who is not saying anything; otherwise, inputs skew toward the loudest, most connected, and most confident. With M&E budgets shrinking, cost-effectiveness becomes a primary driver, making it harder to invest in people-centered approaches that are inherently more expensive.

Lauren tied this back to how Community Listening has shifted since COVID-19. When we treat it as a procurement category, we start optimizing for things that are repeatable, accountable, and easily packaged. Then we start getting into trouble when it comes to inclusion. She agreed that hybrid approaches are important to keep in the mix. 

3)     How do we ensure inclusive and context-responsive tech-enabled community listening? 

Prateek highlighted that Voice AI struggles to effectively reach underrepresented communities, particularly in tribal and remote geographies, because existing models do not adequately capture local dialects, contexts, and access constraints. Data quality for Hindi, for example, is vastly better than for Telugu. AI models have struggled to pronounce people’s names accurately and lacked sufficient context to make sense in some settings. 

Participants explored whether small language models, which can be more context-specific and fine-tuned on local languages, might perform better and help to address language and accessibility barriers. However, accuracy for some languages and dialects remains difficult to address, potentially requiring a large enough corpus to allow an AI interviewer to be understandable, casual, and socially appropriate in a given dialect. This is an area where philanthropic investment could make a real difference – supporting the development of local and small language models and context-specific tools that account for linguistic and cultural diversity.

Closing reflections

Community listening is something valuable in and of itself. It’s a relationship, a willingness to sit down together, to slow down, to trust, to share, to truly hear. The act of listening can help to balance power more equally. 

Community listening is also a process that, in many cases, should lead to action or change. If we don’t close the loop, by sharing results back with the community or acting on what was heard, we risk being extractive and creating frustration and apathy among community members.

Technology can support listening and help with scale, cost, and efficiency. But fully offloading listening to AI or conceiving of community listening as a service we are procuring can risk hollowing out its relational core. We may end up optimizing for the wrong thing entirely.

Finally, before embarking on a community listening process, organizations should honestly assess their readiness. Can the organization meet the community’s pace? This might mean speeding up to make faster and more adaptive decisions, or slowing down if the community wants more time for deeper discussions. Is the organization ready to share power? To shift budget, focus, or timing in response to community feedback? While improving listening mechanisms matters, much of the work that remains is with power holders’ readiness to truly hear and adapt.

AI Disclosure: I wrote this post myself based on the session power point and my notes. I used Claude to lightly edit, instructing it to keep my voice and only flag clunky phrasing, unnecessary repetition, and places my writing was unclear. I took on some of the Claude edits and rejected others or did my own rephrasing. I asked Claude to help me come up with several options for the blog title and chose the 1 I liked the most.