MERL Tech and CLEAR Anglophone Africa hosted three “gLocal evaluation” events in late June under the theme “How to conduct Digital MERL during COVID-19.” These covered the topics of responsible use of data, remote monitoring, and administrative data use.
Key aspects coming out of the events were the need for 1) guidance on data governance and 2) orientation on responsible data practices. Both policy and practice need to be contextualized for the African context and aimed at supporting African monitoring, evaluation, research and learning (MERL) practitioners in their work.
As a follow-on activity, CLEAR Anglophone Africa is calling on M&E practitioners to join up to be a part of this responsible data project for African MERL Practitioners. CLEAR Anglophone Africa and MERL Tech will be collaborating on this responsible data initiative.
Click here to register your interest in the Responsible Data project!
For more information, watch the video from CLEAR Anglophone Africa, and sign up to participate!
Administrative data is that which is collected as part of regular activities that occur during program implementation. It has not been tapped sufficiently for learning and research. As the COVID-19 pandemic advances, how might administrative data be used to help with the COVID response, and other national or global pandemics.
At the final event in the MERL Tech and CLEAR-Anglophone Africa series for gLOCAL Evaluation Week, we were joined by Kwabena Boakye, Ministry of Monitoring and Evaluation, Ghana; Bosco Okumu, National Treasury and Planning, Kenya; Stephen Taylor, Department of Basic Education, South Africa; and Andrea Fletcher, Cooper-Smith.
The four panelists described the kinds of administrative or “routine” data they are using in their work. For example, in Kenya educational records, client information from financial institutions, hospital records of patients, and health outcomes are being used to plan and implement actions related to COVID-19 and to evaluate the impact of different COVID-related policies that governments have put in place or are considering. In Malawi, administrative data is combined with other sources such as Google mobility data to understand how migration might be affecting the virus’ spread. COVID-19 is putting a spotlight on weaknesses and gaps in existing administrative data systems.
Watch the video here:
Listen to just the audio from the event here:
Benefits of administrative data include that:
- Data is generated through normal operations and does not require an additional survey to create it
- It can be more relevant than a survey because it covers a large swath of the entire population
- It is an existing data source during COVID when it’s difficult to collect new data
- It can be used to create dashboards for decision-makers at various levels
- Data sits in silos and the systems are not designed to be interoperable
- Administrative data may leave out those who are not participating in a government program
- Data sets are time-bound to the life of the program
- Some administrative data systems are outdated and have poor quality data that is not useful for decision-making or analysis
- There is a demand for beautiful dashboards and maps but there is insufficient attention to the underlying data processes that would be needed to produce this information so that it can be used
- Real-time data is not possible when there is no Internet connectivity
- There is insufficient attention to data privacy and protection, especially for sensitive data
- Institutions may resist providing data if weakness are highlighted through the data or they think it will make them look bad
Recommendations for better use of administrative data in the public sector:
- Understand the data needs of decision-makers and build capacity to understand and use data systems
- Map the data that exists, assess its quality, and identify gaps
- Design and enact policies and institutional arrangements, tools, and processes to make sure that data is organized and interoperable.
- Automate processes with digital tools to make them more seamless.
- Focus on enhancing underlying data collection processes to improve the quality of administrative data; this includes making it useful for those who provide the data so that it is not yet another administrative burden with no local value.
- Assign accountability for data quality across the entire system.
- Learn from the private sector, but remember that the public sector has different incentives and goals.
- Rather than fund more research on administrative data, donors should put funds into training on data quality, data visualization, and other skills related to data use and data literacy at different levels of government.
- Determine how to improve data quality and use of existing administrative data systems rather than building new ones.
- Make administrative data useful to those who are inputting it to improve data quality.
Download the event reports:
See other gLOCAL Evaluation 2020 events from CLEAR-AA and MERL Tech:
On June 3, MERL Tech and CLEAR-Anglophone Africa hosted the second of three virtual events for gLOCAL Evaluation Week. At this event, we heard from Ignacio Del Busto, IDInsight, Janna Rous, Humanitarian Data, and Ayanda Mtanyana, New Leaders, on the topic of remote monitoring.
Data is not always available, and it can be costly to produce. One challenge is generating data cheaply and quickly to meet the needs of decision-makers within the operational constraints that enumerators face. Another is ensuring that the process is high quality and also human-centered, so that we are not simply extracting data. This can be a challenge when there is low connectivity and reach, poor networks capacity and access, and low smartphone access. Enumerator training is also difficult when it must be done remotely, especially if enumerators are new to technology and more accustomed to doing paper-based surveys.
Watch the video below.
Listen to just the audio from the session here.
Some recommendations arising from the session included:
- Learn and experiment as you try new things. For example, tracking when and why people are dropping off a survey and finding ways to improve the design and approach. This might be related to the time of the call or length of the survey.
- It’s not only about phone surveys. There are other tools. For example, WhatsApp has been used successfully during COVID-19 for collecting health data.
- Don’t just put your paper processes onto a digital device. Instead, consider how to take greater advantage of digital devices and tools to find better ways of monitoring. For example, could we incorporate sensors into the monitoring from the start? At the same time, be careful not to introduce technologies that are overly complex.
- Think about exclusion and access. Who are we excluding when we move to remote monitoring? Children? Women? Elderly people? We might be introducing bias if we are going remote. We also cannot observe if vulnerable people are in a safe place to talk if we are doing remote monitoring. So, we might be exposing people to harm or they could be slipping through the cracks. Also, people self-select for phone surveys. Who is not answering the phone and thus left out of the survey?
- Consider providing airtime but make sure this doesn’t create perverse incentives.
- Ethics and doing no harm are key principles. If we are forced to deliver programs remotely, this involves experimentation. And we are experimenting with people’s lives during a health crisis. Consider including a complaints channel where people can report any issues.
- Ensure data is providing value at the local level, and help teams see what the whole data process is and how their data feeds into it. That will help improve data quality and reduce the tendency to ‘tick the box’ for data collection or find workarounds.
- Design systems for interoperability so that the data can overlap, and the data can be integrated with other data for better insights or can be automatically updated. Data standards need to be established so that different systems can capture data in the same way or the same format;
- Create a well-designed change management program to bring people on board and support them. Role modeling by leaders can help to promote new behaviors.
Further questions to explore:
- How can we design monitoring to be remote from the very start? What new gaps could we fill and what kinds of mixed methods could we use?
- What two-way platforms are most useful and how can they be used effectively and ethically?
- Can we create a simple overview of opportunities and threats of remote monitoring?
- How can we collect qualitative data, e.g., focus groups and in-depth interviews?
- How can we keep respondents safe? What are the repercussions of asking sensitive questions?
- How can we create data continuity plans during the pandemic?
Download the event reports:
See other gLOCAL Evaluation 2020 events from CLEAR-AA and MERL Tech:
Over the past decade, monitoring, evaluation, research and learning (MERL) practices have become increasingly digitalized. The COVID-19 pandemic has caused that the process of digitalization to happen with even greater speed and urgency, due to travel restrictions, quarantine, and social distancing orders from governments who are desperate to slow the spread of the virus and lessen its impact.
MERL Tech and CLEAR-Anglophone Africa are working together to develop a framework and guidance on responsible data management for MERL in the Anglophone African context. As part of this effort, we held three virtual events in early June during CLEAR’s gLOCAL Evaluation Week.
At our June 2 event, Korstiaan Wapenaar, Genesis Analytics, Jerusha Govender, Data Innovator, and Teki Akkueteh, Africa Digital Rights Hub, shared tips on how to be more responsible with data.
Data is a necessary and critical part of COVID-19 prevention and response efforts to understand where the virus might appear next, who is most at risk, and where resources should be directed for prevention and response. However we need to be sure that we are not putting people at risk of privacy violations or misuse of personal data and to ensure that we are managing that data responsibly so that we don’t unnecessarily create fear or panic.
Watch the video below:
Listen to the audio from the session here:
- MERL Practitioners have clear responsibilities when sharing, presenting, consuming and interpreting data. Individuals and institutions may use data to gain prestige, and this can allow bias to creep in or to justify government decisions. Data quality is critical for informing decisions, and information gaps create the risk of misinformation and flawed understanding. We need to embrace uncertainty and the limitations of the science, provide context and definitions so that our sources are clear, and ensure transparency around the numbers and the assumptions that are underpin our work.
- MERL Practitioners should provide contextual information and guidance on how to interpret the data so that people can make sense of it in the right way. We should avoid cherry picking data to prove a point, and we should be aware that data visualization carries power to sway opinions and decisions. It can also influence behavior change in individuals, so we need to take responsibility for that. We also need to find ways to visualize data for lay people and non-technical sectors.
- Critical data is needed, yet it might be used in negative or harmful ways, for example, COVID-related stigmatization that can affect human dignity. We must not override ethical and legal principles in our rush to collect data. Transparency around data collection processes and use are also needed, as well as data minimization. Some might be taking advantage of the situation to amass large amounts of data for alternative purposes, which is unethical. Large amounts of data also bring increased risk of data breaches. When people are scared, such as in COVID times, they will be willing to hand over data. We need to ensure that we are providing oversight and keeping watch over government entities, health facilities, and third-party data processors to ensure data is protected and not misused.
- MERL Practitioners are seeking more guidance and support on: aspects of consent and confidentiality; bias and interference in data collection by governments and community leaders; overcollection of data leading to fatigue; misuse of sensitive data such as location data; potential for re-identification of individuals; data integrity issues; lack of encryption; and some capacity issues.
- Good practices and recommendations include ethical clearance of data and data assurance structures; rigorous methods to reduce bias; third party audits of data and data protection processes; localization and contextualization of data processes and interpretation; and “do no harm” framing.
Read about the other gLOCAL Evaluation 2020 events from CLEAR-AA and MERL Tech: