MERL Tech and Responsible Data: Why we collaborate

This is a cross-post. The original, titled “Responsible Data and MERL” was posted on The Engine Room’s Responsible Data Forum website on February 21, 2022.

Monitoring, evaluation, research and learning (MERL) are by nature data-heavy activities. It makes sense, then, that over the past decade the use of digital technology and digital data has permeated the practice of MERL.

In the social change sphere (humanitarian aid, development work, human rights, and program areas such as health, education, social protection, and protection overall), much of the data that we collect when conducting MERL comes from at-risk or underrepresented populations. This data influences decisions to support these same populations to access rights and services. It also tells us whether or not our interventions have worked and for whom. For this reason, a responsible approach that takes data ethics and data protection into account is an imperative for MERL. 

Around 2013 or so, the use of digital tools and platforms to support MERL began to claim more attention among a small set of ‘early adopter’ MERL practitioners. We began seeing mobile devices used to collect data and to encourage community feedback on programs. The use of satellite data and participatory mapping projects became more prominent, and there was great hope placed on crowdsourcing and citizen journalism for gathering insights. In 2014, Michael Bamberger and I took a closer look, laying out an initial landscape of digital approaches to monitoring and evaluation. 

At the same time, following a discussion on the ethics of participatory mapping, a group of practitioners (including The Engine Room’s founders) assembled to look more closely at ethics in technology for development. We were concerned that development agencies were pushing innovation and technology in development while being largely unaware of data ethics, privacy and security issues that could expose individuals and communities to risk. Our first MERL Tech Conference also happened that year.

When adoption of innovation is used as the principal indicator of success or failure, the wider positive or negative ramifications, (including unintended consequences, costs, and risks) are likely to be overlooked, as noted by Glover. Close overlap with the Responsible Data community has helped ensure that the MERL Tech community continuously reflects on these concerns. We have created a space for dialog and discussion between tech developers, early adopters of technology in MERL, privacy and ethics advocates, and those who are newly learning about how technology can enable MERL. This has allowed the sector to improve its use of digital data, to highlight potential negative outcomes for vulnerable groups, and to introduce approaches to mitigate the harm that can come from the collection and use of MERL data.

The Responsible Data (RD) and MERL Tech communities have together explored many such areas, including consent in the digital age, developing responsible data policies, and operationalizing responsible data policies. As the MERL community matured, discussions about RD moved from the margins to the center.

Over the years, we’ve seen heated debates about the potential for harm and unintended consequences stemming from digital approaches or from partnerships whose value and ethics are questioned. Together, the MERL Tech and RD communities advocated for ethical frameworks and better research on the potential for technologies and digital approaches to do harm. For example, research by Oxfam and The Engine Room on balancing the risks and benefits of using biometrics in the humanitarian field has helped to shape how we think about the role of emerging technology in MERL.

We also co-curate a Responsible Data Resource List which lives on both MERL Tech and RD websites. Key to these discussions is the active and lively community debate that happens on the RD listserv,  at events by both organizations, and discussions that have taken place on these themes at the New York City Technology Salons. This constant reflection on the responsibilities of MERL professionals to use data in ethical and responsible ways has strengthened the MERL Tech space. 

While collective progress is being made in documenting and assessing technology-enabled MERL initiatives and good practice guidelines are available, ethical questions remain about these new and emerging methods and approaches. In 2022, we are talking about much more than mobile data gathering, mapping and crowdsourcing, as we have explored in a series of MERL Tech State of the Field reports. Alongside more traditional uses of digital tools and data, we see ‘big data’ and predictive analytics sitting squarely in the MERL space. Emerging tools and technologies such as blockchain, artificial intelligence and machine learning, new forms of data storage, text and voice analytics, biometrics, non-traditional data and metadata are being explored as part of the MERL toolbox. Collaboration with the RD community helps the MERL Tech community ensure that we are looking at the ethics and RD issues that come with these evolving approaches. 

Going forward, the MERL Tech community is addressing the fallout of COVID-19, which has made digital MERL and remote monitoring even more relevant. Building on a Responsible Data in MERL during COVID-19 series of events co-hosted with the CLEAR Center in Anglophone Africa, we convened a Responsible Data in M&E – RDiME working group and community of practice that focus on these issues in the African context. Members of the working group developed guidance on data governance and responsible data practices for MERL with a focus on African contexts. We have also continued our focus on documenting, sharing, learning, training, and guidance on how to improve rigor, validity, representativeness and inclusion, how to mitigate potential harms to vulnerable individuals and groups, and how to assess new approaches and methods to ensure that ethics and safeguarding are included in MERL design and implementation. 

Ultimately, we hope our two communities will continue to collaborate in order to strengthen the MERL sector through intentional, responsible, and ethical approaches to technology and digital data.