Tag Archives: London

Technologies in monitoring and evaluation | 5 takeaways

Bloggers: Martijn Marijnis and Leonard Zijlstra. This post originally appeared on the ICCO blog on April 3, 2018.
.

Technologies in monitoring and evaluation | 5 takeaways

On March 19 and 20 ICCO participated in the MERL Tech 2018 in London. The conference explores the possibilities of technology in monitoring, evaluation, learning and research in development. About 200 like-minded participants from various countries participated. Key issues on the agenda were data privacy, data literacy within and beyond your organization, human-centred monitoring design and user-driven technologies. Interesting practices where shared, amongst others in using blockchain technologies and machine learning. Here are our most important takeaways:

1)  In many NGOs data gathering still takes place in silo’s

Oxfam UK shared some knowledgeable insights and practical tips in putting in place an infrastructure that combines data: start small and test, e.g. by building up a strong country use case; discuss with and learn from others; ensure privacy by design and make sure senior leadership is involved. ICCO Cooperation currently faces a similar challenge, in particular in combining our household data with our global result indicators.

2)  Machine learning has potential for NGOs

While ICCO recently started to test machine learning in the food security field (see this blog) other organisations showcased interesting examples: the Wellcome Trust shared a case where they tried to answer the following question: Is the organization informing and influencing policy and if so, how? Wellcome teamed up their data lab and insight & analysis team and started to use open APIs to pull data in combination with natural language processing to identify relevant cases of research supported by the organization. With their 8.000 publications a year this would be a daunting task for a human team. First, publications linked to Wellcome funding were extracted from a European database (EPMC) in combination with end of grant reports. Then WHO’s reference section was scraped to see if and to what extent WHO’s policy was influenced and to identify potential interesting cases for Wellcome’s policy team.

3)  Use a standardized framework for digital development

See digitalpinciples.org. It gives – amongst others – practical guidelines on how to use open standards and open data, how data can be reused, how privacy and security can be addressed, how users can and should be involved in using technologies in development projects. It is a useful framework for evaluating your design.

4)  Many INGOs get nervous these days about blockchain technology

What is it, a new hype or a real game changer? For many it is just untested technology with high risks and little upside for the developing world. But, for example INGOs working in agriculture value chains or in humanitarian relief operations, its potential is definitely consequential enough to merit a closer look. It starts with the underlying principle, that users of a so-called blockchain can transfer value, or assets, between each other without the need for a trusted intermediary. The running history of the transactions is called the blockchain, and each transaction is called a block. All transactions are recorded in a ledger that is shared by all users of a blockchain.

The upside of blockchain applications is the considerable time and money saving aspect of it. Users rely on this shared ledger to provide a transparent view into the details of the assets or values, including who owns them, as well as descriptive information such as quality or location. Smallholder farmers could benefit (e.g. real-time payment on delivery, access to credit), so can international sourcing companies (e.g. traceability of produce without certification), banks (e.g. cost-reductions, risk-reduction), as much as refugees and landless (e.g. registration, identification). Although we haven’t yet seen large-scale adoption of blockchain technology in the development sector, investors like the Bill and Melinda Gates Foundation and various venture capitalists are paying attention to this space.

But one of the main downsides or challenges for blockchain, like with agricultural technology at large, is connecting the technology to viable business models and compelling use cases. With or without tested technology, this is hard enough as it is and requires innovation, perseverance and focus on real value for the end-user; ICCO’s G4AW projects gain experience with blockchain.

5)  Start thinking about data-use incentives

Over the years, ICCO has made significant investments in monitoring & evaluation and data skills training. Yet limited measurable results of increased data use can be seen, like in many other organizations. US-based development consultant Cooper&Smith shared revealing insights into data usage incentives. It tested three INGOs working across five regions globally. The hypothesis was, that better alignment of data-use training incentives leads to increased data use later on. They looked at both financial and non-financial rewards that motivate individuals to behave in a particular way. Incentives included different training formats (e.g. individual, blended), different hardware (e.g. desktop, laptop, mobile phone), recognition (e.g. certificate, presentation at a conference), forms of feedback & support (e.g. one-on-one, peer group) and leisure time during the training (e.g. 2 hours/week, 12 hours/week). Data use was referred to as the practice of collecting, managing, analyzing and interpreting data for making program policy and management decisions.

They found considerable differences in appreciation of the attributes. For instance, respondents overwhelmingly prefer a certificate in data management, but instead they currently receive primarily no recognition or only recognition from their supervisor. Or  one region prefers a certificate while the other prefers attending an international conference as reward. Or that they prefer one-on-one feedback but instead they receive only peer-2-peer support. The lesson here is, that while most organizations apply a ‘one-size fits all’-reward system (or have no reward system at all), this study points at the need to develop a culturally sensitive and geographically smart reward system to see real increase in data usage.

For many NGOs the data revolution has just begun, but we are underway!

MERL Tech London 2018 Agenda is out!

We’ve been working hard over the past several weeks to finish up the agenda for MERL Tech London 2018, and it’s now ready!

We’ve got workshops, panels, discussions, case studies, lightning talks, demos, community building, socializing, and an evening reception with a Fail Fest!

Topics range from mobile data collection, to organizational capacity, to learning and good practice for information systems, to data science approaches, to qualitative methods using mobile ethnography and video, to biometrics and blockchain, to data ethics and privacy and more.

You can search the agenda to find the topics, themes and tools that are most interesting, identify sessions that are most relevant to your organization’s size and approach, pick the session methodologies that you prefer (some of us like participatory and some of us like listening), and to learn more about the different speakers and facilitators and their work.

Tickets are going fast, so be sure to snap yours up before it’s too late! (Register here!)

View the MERL Tech London schedule & directory.

 

MERL Tech London session ideas due this Friday, Nov 10th!

MERL Tech London is coming up on March 19-20, 2018. Session ideas are due by Friday, November 10th, so be sure to get yours in this week!!

Submission Deadline: Friday, November 10, 2017.

Session leads receive priority for the available seats at MERL Tech and a discounted registration fee. You will hear back from us in early December and, if selected, you will be asked to submit an updated and final session title, summary and outline by January 19th, 2018.

Topics we’re looking for:

  • Case studies: Sharing end-to-end experiences/learning from a MERL Tech process
  • MERL Tech 101: How-to use a MERL Tech tool or approach
  • Methods & Frameworks: Sharing/developing/discussing methods and frameworks for MERL Tech
  • Data: Big, large, small, quant, qual, real-time, online-offline, approaches, quality, etc.
  • Innovations: Brand new, untested technologies or approaches and their application to MERL(Tech)
  • Debates: Lively discussions, big picture conundrums, thorny questions, contentious topics related to MERL Tech
  • Management: People, organizations, partners, capacity strengthening, adaptive management, change processes related to MERL Tech
  • Evaluating MERL Tech: comparisons or learnings about MERL Tech tools/approaches and technology in development processes
  • Failures: What hasn’t worked and why, and what can be learned from this?
  • Demo Tables: to share MERL Tech approaches, tools, and technologies
  • Other topics we may have missed!

To get you thinking — take a look at past agendas from MERL Tech LondonMERL Tech DC and MERL Tech News.

Submit your session idea now!

We’re actively seeking a diverse (in every way) set of MERL Tech practitioners to facilitate every session. We encourage organizations to open this opportunity to colleagues and partners working outside of headquarters and to support their participation. (And please, no all-male panels!)

MERL Tech is dedicated to creating a safe, inclusive, welcoming and harassment-free experience for everyone. Please review our Code of Conduct. Session submissions are reviewed by our steering committee.

Submit your session ideas by November 10th!

If you have any questions about your submission idea, please contact Linda Raftree.

(Registration is also open!)

Submit your session ideas for MERL Tech London by Nov 10th!

MERL Tech London

Please submit a session idea, register to attend, or reserve a demo table for MERL Tech London, on March 19-20, 2018, for in-depth sharing and exploration of what’s happening across the multidisciplinary monitoring, evaluation, research and learning field.

Building on MERL Tech London 2017, we will engage 200 practitioners from across the development and technology ecosystems for a two-day conference seeking to turn the theories of MERL technology into effective practice that delivers real insight and learning in our sector.

MERL Tech London 2018

Digital data and new media and information technologies are changing MERL practices. The past five years have seen technology-enabled MERL growing by leaps and bounds, including:

  • Adaptive management and ‘developmental evaluation’
  • Faster, higher quality data collection.
  • Remote data gathering through sensors and self-reporting by mobile.
  • Big Data and social media analytics
  • Story-triggered methodologies

Alongside these new initiatives, we are seeing increasing documentation and assessment of technology-enabled MERL initiatives. Good practice guidelines and new frameworks are emerging and agency-level efforts are making new initiatives easier to start, build on and improve.

The swarm of ethical questions related to these new methods and approaches has spurred greater attention to areas such as responsible data practice and the development of policies, guidelines and minimum ethical frameworks and standards for digital data.

Please submit a session idea, register to attend, or reserve a demo table for MERL Tech London to discuss all this and more! You’ll have the chance to meet, learn from, debate with 150-200 of your MERL Tech peers and to see live demos of new tools and approaches to MERL.

Submit Your Session Ideas Now!

Like previous conferences, MERL Tech London will be a highly participatory, community-driven event and we’re actively seeking practitioners in monitoring, evaluation, research, learning, data science and technology to facilitate every session.

Please submit your session ideas now. We are particularly interested in:

  • Case studies: Sharing end-to-end experiences/learning from a MERL Tech process
  • MERL Tech 101: How-to use a MERL Tech tool or approach
  • Methods & Frameworks: Sharing/developing/discussing methods and frameworks for MERL Tech
  • Data: Big, large, small, quant, qual, real-time, online-offline, approaches, quality, etc.
  • Innovations: Brand new, untested technologies or approaches and their application to MERL(Tech)
  • Debates: Lively discussions, big picture conundrums, thorny questions, contentious topics related to MERL Tech
  • Management: People, organizations, partners, capacity strengthening, adaptive management, change processes related to MERL Tech
  • Evaluating MERL Tech: comparisons or learnings about MERL Tech tools/approaches and technology in development processes
  • Failures: What hasn’t worked and why, and what can be learned from this?
  • Demo Tables: to share MERL Tech approaches, tools, and technologies
  • Other topics we may have missed!

Session Submission Deadline: Friday, November 10, 2017.

Session leads receive priority for the available seats at MERL Tech and a discounted registration fee. You will hear back from us in early December and, if selected, you will be asked to submit an updated and final session title, summary and outline by Friday, January 19th, 2018.

Register Now!

Please register to attend, or reserve a demo table for MERL Tech London 2018 to examine these trends with an exciting mix of educational keynotes, lightning talks, and group breakouts, including an evening Fail Festival reception to foster needed networking across sectors.

We are charging a modest fee to better allocate seats and we expect to sell out quickly again this year, so buy your tickets or demo tables now. Event proceeds will be used to cover event costs and to offer travel stipends for select participants implementing MERL Tech activities in developing countries.

Technology in MERL: an approximate history

by Linda Raftree, MERL Tech co-organizer.

At MERL Tech London, Maliha Khan led us in an exercise to map out our shared history of MERL Tech. Following that we did some prioritizing around potential next steps for the sector (which I’ll cover in a next post).

She had us each write down 1) When we first got involved in something related to MERL Tech, and 2) What would we identify as a defining moment or key event in the wider field or in terms of our own experiences with MERL Tech.

The results were a crowdsourced MERL Tech Timeline on the wall.

 

An approximate history of tech in MERL 

We discussed the general flow of how technology had come to merge with MERL in humanitarian and development work over the past 20 years. The purpose was not to debate about exact dates, but to get a sense of how the field and community had emerged and how participants had experienced its ebbs and flows over time.

Some highlights:

  • 1996 digital photos being used in community-led research
  • 1998 mobile phones start to creep more and more into our work
  • 2000 the rise of SMS
  • 2001 spread of mobile phone use among development/aid workers, especially when disasters hit
  • 2003 Mobile Money comes onto the scene
  • 2004 enter smart phones; Asian tsunami happens and illustrates need for greater collaboration
  • 2005 increased focus on smartphones; enter Google maps
  • 2008 IATI, Hans Rosling interactive data talk/data visualization
  • 2009 ODK, FrontlineSMS, more and more Mobile Money and smart phones, open data; global ICT4D conference
  • 2010 Haiti earthquakes – health, GIS and infrastructure data collected at large scale, SMS reporting and mapping
  • 2011 FrontlineSMS’ data integrity guide
  • 2012 introduction and spread of cloud services in our work; more and more mapping/GIS in humanitarian and development work
  • 2013 more focus and funding from donors for tech-enabled work, more awareness and work on data standards and protocols, more use of tablets for data collection, bitcoin and blockchain enter the humanitarian/development scene; big data
  • 2014 landscape report on use of ICTs for M&E; MERL Tech conference starts to come together; Responsible Data Forum; U-Report and feedback loops; thinking about SDGs and Data revolution
  • 2015 Ebola crisis leads to different approach to data, big data concerns and ‘big data disasters’, awareness of the need for much improved coordination on tech and digital data; World Bank Digital Dividends report; Oxfam Responsible Data policy
  • 2016 real-time data and feedback loops are better unpacked and starting to be more integrated, adaptive management focus, greater awareness of need of interoperability, concerns about digital data privacy and security
  • 2017 MERL Tech London and the coming-together of the related community

What do you think? What’s missing? We’d love to have a more complete and accurate timeline at some point….