Tag Archives: real-time data

Using Real-Time Data to Improve International Development Programming

by Erica Gendell, Program Analyst at USAID; and Rebecca Saxton-Fox, ICT Policy Advisor at USAID

Real-time data applications in international development

There are a wide range of applications of real-time data in international development programs, including:

  • Gathering demographic and assessment data following trainings, in order to improve outputs and outreach for future trainings;
  • Tracking migration flows following natural disasters to understand population locations and best locate relief efforts;
  • Analyzing real-time disease outbreak data to understand where medical resources will be most effectively deployed; and
  • Analyzing of radio and social media to understand and adapt communication outreach.

Using digital tools (such as mobile phone based text messaging, web-based applications, social media platforms, etc.) or large digital datasets (such as satellite or cell phone tower data) for collecting real-time data helps programs and projects respond quickly to community needs or potentially changing circumstances on the ground. However, these digital tools and datasets are often not well understood or mapped into decision-making processes.

Real Example of Real-time Data

In USAID/Ghana’s ADVANCE II program, project staff implemented a smart card ID technology that collects and stores data in an effort to have more accurate monitoring and evaluation data on project beneficiaries. The ID cards allowed USAID and project officers to see real-time results and build more effective and targeted programming. ADVANCE II has been successful in providing unique beneficiary data for over 120,000 people who participated in 5,111 training sessions. This information enabled the project to increase the number of trainings tailored to female farmers, a previously underrepresented population in trainings. This is a great example of how to incorporate data use and digital tools into a project or activity.

Data to Action Framework

At MERL Tech DC, we presented the ADVANCE II project as a way to use the “Data to Action” Framework. This is one approach to map how information flows and how decisions are made across a set of stakeholders in a program. It can be used as a conversation tool to identify barriers to action. You can also use it to identify where digital tools could help move information to decision makers faster.

This framework is just one tool to start thinking about uses of real-time data to enable adaptive management in development programs.

USAID explores these and other topics in a newly released portfolio of research on Real-time Data for Adaptive Management (RTD4AM), which give insight into the barriers to real-time data use in development. We look forward to continuing to build the community of practice of adaptive management within the MERL community.

 

 

Moving from “evaluation” to “impact management”

by Richa Verma, Resident Entrepreneur at Social Cops. This post originally appeared on the Social Cops blog on August 28, 2017.

When I say that Impact Evaluation is history, I mean it. Some people will question this. After all, Impact Evaluation just became mainstream in the last decade, driven by great improvements in experimental design methods like randomized control trials (RCTs). So how can I say that it’s already a thing of the past? It’s not Impact Evaluation’s fault. The world changed.

Methodologies like RCTs came from medical science, where you can give patients a pill and assess its impact with randomized trials. However, development is not a space where one pill will work for everyone. In development, the patients change faster, the illness evolves faster, and the pill needs to keep pace with both the patients and the illness. That’s where Impact Management comes in.

What Is Impact Management?

New Philanthropy Capital‘s 2017 Global Innovation in Measurement and Evaluation Report counts Impact Management as one of the top 7 innovations of 2017.

So what is Impact Management? Let me first explain what it is not. It’s not a one-time evaluation. It’s not collecting data for answering a limited set of questions. It’s not a separate activity from your program. It’s not just monitoring and evaluation.

It’s a way of making data-driven decisions at every step of your program. It’s about keeping a pulse on your program every day and finding new questions to answer, rather than just focusing on specific questions predetermined by your monitoring and evaluation team or funders.

“The question that’s being asked more and more is, ‘How does evaluation feed into better management decisions?’ That’s a shift from measurement of impact, to measurement for impact.”
– Megan Campbell (Feedback Labs)

How Does Impact Management Work?

Impact Management uses the basic components of monitoring and evaluation, but with an outlook shift. It involves frequent data collection, regular reporting and monitoring of your data, and iteratively updating your program indicators and metrics as data comes in and the program changes.

Impact Management differs from Impact Assessment in that it promotes course correction on a daily basis. Organizations collect data on their programs as they conduct activities, analyze that information on a regular basis, and make changes to the program.

With an outlook that encourages frequent changes, as if you were trading in stocks, organizations will have the ability to A/B test their programs with real-time data to make decisions immediately; rather than wait to compare and contrast two different surveys. They can test out new things and make changes as they receive data in servers, even at the end of the day rather than waiting for the official year-end review. It becomes a way of deciding how they should execute a program daily rather than only seeing strategic changes through.

“[Data collection] should be ongoing — it’s a value driver not a compliance requirement.”
– Tom Adams (Acumen)

In many ways, this is how decisions are made on Wall Street or Dalal Street in India. Analysts don’t wait until the end of the year to make investments by reviewing annual reports. They watch daily as the market fluctuates and strike as soon as they see new potential.

Impact Management works exactly the same. You should strive to increase your impact as soon as opportunity arrives, rather than waiting for a year-end external evaluation or approval.

How Can You Implement Impact Management?

To make Impact Management possible, switch from static data files to a flexible data system.

Today, most of your program officers and even your beneficiaries are armed with mini-computers in their pockets (read: smartphones). Leverage these to create a network of data ingestion devices, continuously tracking and measuring the impact of your programs. Use mobile data collection apps to add forms, deploy them to the field, and reach out not just to your field force but also your beneficiaries — not just at the end of the month or quarter, but as frequently as possible.

Then don’t let this data sit in Excel files. Use today’s technologies to create your own data management system, one that will link your beneficiaries, connect your programs, and answer queries. Have someone with an analytical bent look at this data regularly, or draw on machine power to analyze this data and generate meaningful insights or reports in real time.

“We’re moving away from a static data world, where you work on datasets, and you write reports, to a dynamic data world where data is always being generated and created and it helps you do your job better.”
– Andrew Means (beyond.uptake)

Lastly, it’s crucial to tie this flexible data system back to your decisions. Make real-time data — rather than guesses or last year’s data — the basis of every program decision and the foundation of even weekly catch-ups. And don’t hesitate to test out new things. Data will tell you whether something worked or not.

Many of our partners are using our platform to make Impact Management possible and track their programs in real time. The platform lets them create and tweak data collection forms, and monitor incoming data in real time on their computer, in regular reports, or even on map-based dashboards. They are asking new questions about how their programs are doing and answering them with data.

If we really want to create the best development programs, we’ll have to think differently and use evidence not just once every month or year, but as we make crucial decisions every day. All backed by the tenets of Impact Management: test, fail, improve, repeat.

Join us at MERL Tech London on March 19-20 – where we’ll be debating this topic!