Tag Archives: collection

Emerging Technologies: How Can We Use Them for MERL?

Guest post from Kerry Bruce, Clear Outcomes

A new wave of technologies and approaches has the potential to influence how monitoring, evaluation, research and learning (MERL) practitioners do their work. The growth in use of smartphones and the internet, digitization of existing data sets, and collection of digital data make data increasingly available for MERL activities. This changes how MERL is conducted and, in some cases, who conducts it.

We recently completed research on emerging technologies for use in MERL as part of a wider research project on The State of the Field of MERL Tech.

We hypothesized that emerging technology is revolutionizing the types of data that can be collected and accessed and the ways that it can be processed and used for better MERL. However, improved research on and documentation of how these technologies are being used is required so the sector can better understand where, when, why, how, and for which populations and which types of MERL these emerging technologies would be appropriate.

The team reviewed the state of the field and found there were three key new areas of data that MERL practitioners should consider:

  • New kinds of data sources, such as application data, sensor data, data from drones and biometrics. These types of data are providing more access to information and larger volumes of data than ever before.
  • New types of systems for data storage.  The most prominent of these was the distributed ledger technologies (also known as blockchain) and an increasing use of cloud and edge computing.  We discuss the implications of these technologies for MERL.
  • New ways of processing data, mainly from the field of machine learning, specifically supervised and unsupervised learning techniques that could help MERL practitioners manage large volumes of both quantitative and qualitative data.

These new technologies hold great promise for making MERL practices more precise, automated and timely. However, some challenges include:

  • A need to clearly define problems so the choice of data, tool, or technique is appropriate
  • Non-representative selection bias when sampling
  • Reduced MERL practitioner or evaluator control
  • Change management needs to adapt how organizations manage data
  • Rapid platform changes and difficulty with assessing the costs
  • A need for systems thinking which may involve stitching different technologies together

To address emerging challenges and make best use of the new data, tools, and approaches, we found a need for capacity strengthening for MERL practitioners, greater collaboration among social scientists and technologists, a need for increased documentation, and a need for the incorporation of more systems thinking among MERL practitioners.

Finally there remains a need for greater attention to justice, ethics and privacy in emerging technology.

Download the paper here!

Read the other papers in the series here!

Collecting Data in Hard to Reach Places

Written by Stephanie Jamilla

By virtue of operating in the international development sphere, we oftentimes work in areas that are remote, isolated, and have little or no internet connection. However, as the presenters from Medic Mobile and Vitamin Angels (VA) argued in their talk, “Data Approaches in Hard-to-Reach Places,” it is possible to overcome these barriers and use technology to collect much-needed program data. The session was split neatly into three parts: a presentation by Mourice Barasa, the Impact Lead of Medic Mobile in Kenya, a presentation by Jamie Frederick, M&E Manager, and Samantha Serrano, M&E Specialist, from Vitamin Angels, and an activity for attendees. 

While both presentations discussed data collection in a global health context and used phone applications as the means of data collection, they illustrated two different situations. Barasa focused on the community health app that Medic Mobile is implementing. It is used by community health teams to better manage their health workers and to ease the process of providing care. The app serves many purposes. For example, it is a communication tool that connects managers and health workers as well as a performance management tool that tracks the progress of health workers and the types of cases they have worked on. The overall idea is to provide near real time (NRT) data so that health teams have up-to-date information about who has been seen, what patients need, if patients need to be seen in a health facility, etc. Medic Mobile implemented the app with the Ministry of Health in Siaya, Kenya and currently have 1700 health workers using the tools. While the use of the app is impressive, Barasa explained various barriers that hinder the app from creating NRT data. Health teams rely on the timestamp sent with every entry to know when a household is visited by a health worker. However, a health worker may wait to upload an entry and use the default time on their phone rather than the actual time of visit. Also, poor connectivity, short battery life, and internet subscription costs are of concern. Medic Mobile is working on improvements such as exploring the possibility of using offline servers, finding alternatives to phone charging, and central billing of mobile users have decreased billing from $2000/month to around $100.

Frederick and Serrano expressed similar difficulties in their presentation — particularly about the timeliness of data upload. However, their situation was different. VA used their app for specifically M&E purposes. The organization wanted to validate the extent to which it was reaching its target population, delivering services at the best practice standard, and are truly filling the 30% gap of coverage that national health services miss. Their monitoring design consisted of taking a random sample of 20% of their field partners and using ODK collect with an ONA-programmed survey (which is cloud-based) on Android devices. VA trained 30 monitors to cover countries in Latin America and the Caribbean, Africa, and Asia in which they had partners. While the VA Home Office was able to use the data collected on the app well through the cycle of data collection to action, field partners were having trouble with the data in the analysis, reporting, and action stages. Hence, a potential solution was piloted with three partners in Latin America. VA adjusted the surveys in ONA so that it would display a simple report with internal calculations based on the survey data. This report was developed in NRT, allowing partners to access the data quickly. VA also formatted the report so that the data was easily consumable. VA also made sure to gather feedback from partners about the usefulness of monitoring results to ensure that partners also valued collecting this data. 

These two presentations reinforced that while there is the ability to collect data in difficult places, there will always be barriers as well, whether they are technical or human-related. The group discussion activity revealed other challenges. The presenters prompted the audience with four questions:

  1. What are data collection approaches you have used in hard-to-reach places?
  2. What have been some challenges with these approaches?
  3. How have the data been used?
  4. What have been some challenges with use of these data?

In my group of five, we talked mainly about hindrances to data collection in our own work, such as the cost of some technology. Another that came up was how there is a gap between having the data visualizing them well but ensuring that the data we do collect actually translates into action.

Overall, the session helped me think through how important it is to consider potential challenges in the initial design of the data collection and analysis process. The experiences of Medic Mobile and Vitamin Angels demonstrated what difficulties we all will face when collecting data in hard-to-reach places but also that those difficulties can ultimately be overcome.