Tuesday 11.15-13.15 (Bridewell Hall)
Real time data for MERL: Joys and challenges of making machines do the work: Sam Moody and Hilary Cornish, Christian Aid, Girija Bahety, Oxford Policy Management
Are you interested in or already using real time data collection and analysis for programme delivery, monitoring or evaluations? Technology that does your data crunching automatically can take work off our plates and limit the space for human error. The potential uses are myriad but what about the risks? This session gives practical examples, interactive tasks and open discussion to explore some of these.
What are the opportunities and risks associated with using real-time data in collection and analysis? Are there situations where the process can be automated to the extent that decisions themselves can be automated? What if we want to use it to help us to develop an understanding of whether and how change is happening? Oxford Policy Management (OPM) has used the former approach, whilst Christian Aid (CA) has attempted the latter, both as part of wider processes. In both cases, steps in the process that might previously have been carried out by human beings are now being done by machines.
This session explores the opportunities and risks emerging from increased use of real time data and automated data processing. It will highlight the key challenges through an interactive data gathering exercise, and hands-on experiences of rapid analysis. Participants will leave the session with a better understanding of how real-time automated approaches can support programme delivery and evaluation, and a better understanding of the risks it can pose.
The session will begin with short presentations from CA and OPM, giving cases of where automated data collection, monitoring and analysis has been used in the field. In Bihar, India OPM conducted real-time monitoring of data to automatically generate lists of women who had met the conditions to receive a cash transfer that month. They used the data to inform the coverage of the programme as well as the services available and received in the programme areas. CA will draw on an evaluation of a complex resilience programme in Kenya, where interactive dashboards of survey data were used to inform simultaneously conducted focus groups through ‘on the fly’ analysis of the incoming data.
The workshop will then move into a hands-on exercise, giving practical experience of the use of instant survey data visualisations in evaluation. In an imaginary scenario, participants will collect data, and add it to an existing dataset. In small groups you will use visualisations of the data to offer an early analysis, and highlight initial areas to explore. We’ll work together to explore the limits of this analysis, and highlight the challenges emerging with this way of working with real time data. We’ll end the session with a group discussion, drawing on participants’ own experiences of using real time data analysis, looking at the key benefits and key concerns and how to address them.
Girija Bahety is a Consultant in the Social Policy Programme at the Oxford Policy Management focussing on monitoring and evaluation in the social sectors of social protection, health and nutrition, based out of Oxford (UK). She has been involved in design, implementation, and impact evaluation of DfID-funded Bihar Child Support Program, a conditional cash transfer programme in Bihar (India) aimed at improving nutritional status of mothers and children under 3 years. In particular, she has designed and operationalised a JAVA and android based mobile phone application for real-time monitoring of the programme used by the frontline health workers. She has also been involved in several impact evaluations, process evaluations and monitoring system projects across Malawi, Nigeria, India, Pakistan and Nepal. Girija has completed an MPhil in Economics from the University of Cambridge, where she was a Chevening Scholar. She completed her Bachelors in Economics from the University of Delhi.
Sam Moody is Data and Transparency Advisor at Christian Aid and is based in London. In his position, he assesses the way that the organisation gathers, manages, analyses and presents information, looking for opportunities to improve impact. This includes work on digital MERL, internal decision-making processes and increased accountability through meaningful access to data. His primary interest in this work is in the extent to which digital technology can democratise the process of learning from data, shifting power dynamics by removing the need to rely on small numbers of people to interpret data for you. Before joining Christian Aid, he managed a community project in Gambella, Ethiopia and then lived on the Thailand-Myanmar border building the internal capacity of a local human rights organisation. More about Sam here.
Hilary Cornish is a Research, Evidence and Learning Adviser at Christian Aid, where she works as part of a new team which focusses on strengthening research and evidencing across the organisation through strategic interventions, capacity building, resource development and direct support to specific research and evaluation projects. She has an academic research background, with a PhD from the University of Edinburgh looking at culture and language learning in conflict. She has a passion for using digital methods to support research, including data collection, communication and uptake. More about Hilary here.
Listen to Sam’s presentation on using real-time monitoring of survey data to improve the quality of evaluation: