Early Concepts for Designing and Evaluating Blockchain Interventions for Behavior Change
Guest post by Michael Cooper, a former DoS, MCC Associate Director for Policy and Evaluation who now runs Emergence. Mike advises numerous donors, private clients and foundations on program design, MEL, adaptive management and other analytical functions.
International development projects using the blockchain in some way are increasing at a rapid rate and our window for developing evidence around what does and does not work (and more importantly why) is narrow before we run into un-intended consequences. Given that blockchain is a highly disruptive technology, these un-intended consequences could be significant, creating a higher urgency to generate the evidence to guide how we design and evaluate blockchain applications.
Our window for developing evidence around what does and does not work (and more importantly why) is narrow before we run into un-intended consequences.
To inform this discussion, Emergence has put out a working paper that outlines 1.) what the blockchain is, 2.) how it can be used to leverage behavior change outcomes in international development projects and 3.) the implications for how we could design and evaluate blockchain based interventions. The paper utilizes systems and behaviorism principles in comparing how we currently design behavior change interventions to how we could design/evaluate the same interventions using the blockchain. This article summarizes the main points of the paper and its conclusions to generate discussion around how to best produce the evidence we need to fully realize the potential of blockchain interventions for social impact.
Given the scope of possibilities surrounding the blockchain, both in how it could be used and in the impact it could leverage, the implications for how MEL is conducted are significant. The time is long gone where value adding MEL practitioners are not involved in intervention design. Blockchain based interventions will require additional integration of MEL skill sets in the early design phases since so much will need to be “tested” to determine what is and is not working. While rigid statistical evaluations will needed for some of these blockchain based interventions, the level of complexity involved and the lack of an evidence base indicate that more flexible, adaptive and more formative MEL approaches will be needed. The more these approaches are proactive and involved in intervention design, the more frequent and informative the feedback loops will be into our evidence base.
The Blockchain as a Decentralizing Technology
At its core, the blockchain is just a ledger but the importance of ledgers in how society functions cannot be understated. Ledgers, and the control of them, are crucial in how supply chains are managed, financial transactions are conducted, how data is shared, etc. Control of ledgers is a primary factor in limiting access to life changing goods and services, especially for the worlds’ poor. In part, the discussion over decentralization is essentially a discussion over who owns and how ledgers are managed.
Decentralization has been a prominent theme in international development and there is strong evidence of its positive impact across various sectors, especially regarding local service delivery. One of the primary value adds of decentralization is empowering those further from traditional concentrations of power to have more authority over the problems that impact them. As a decentralizing technology, the blockchain holds a lot of potential in reaching these same impacts from decentralization (empowerment, etc.) in a more efficient and effective manner partly due to its ability to better align interests around common problems. With better aligned interests, less resources (inputs) are needed to try and facilitate a desired behavior change.
Up until now, efforts of international development actors have focused on “nudging” behavior change amongst stakeholders and in very rare cases, such as in results based financing, give loosely defined parameters to implementers with less emphasis on the manner in which outcomes are achieved. Both of these approaches are relevant in the design and testing of blockchain based interventions but they will be integrated in unique new ways that will require new thinking and skills sets amongst practitioners.
Current Designing and Evaluating for Behavior Change
MEL usually starts with the relevant theory of change, namely what mechanisms bring about targeted behavior change and how. Recent years have seen a focus on how behavior change is achieved through an understanding of mindsets and how they can be nudged to achieve a social outcome. However the international development space has recognized the limitations of designing interventions that attempt to nudge behavior change. These limitations center around the level of complexity involved, the inability to recognize and manage this complexity and lack of awareness about the root causes of problems. Hence the rise in things like results based financing where the type of prescribed top-down causal pathway (usually laid out in a theory of change) is not as heavily emphasized as in more traditional interventions. Donors using this approach can still mandate certain principles of implementation (such as the inclusion of vulnerable populations, environmental safeguards, timelines, etc.) but there is much more flexibility to create a causal pathway to achieve the outcome.
Or, for example, take the popular PDIA approach where the focus is on iteratively identifying and solving problems encountered on the pathway to reform. These efforts do not start with a mandated theory of change, but instead start with generally described targeted outcomes and then the pathway to those outcomes is iteratively created, similar to what Lant Pritchett has called “crawling the design space”. Such an approach has large overlaps with adaptive management practices and other more integrative MEL frameworks and could lend themselves to how blockchain based interventions are designed, implemented and evaluated.
How the Blockchain Could Achieve Outcomes and Implications for MEL
Because of its decentralizing effects, any theory of change for a blockchain based intervention could include some possible common attributes that influence how outcomes are achieved:
- Empowerment of those closest to problems to inform the relevant solutions
- Alignment of interests around these solutions
- Alleviation of traditional intermediary services and relevant third party actors
Assessing these three attributes, and how they influence outcomes, could be the foundation of any appropriate MEL strategy for a blockchain-based intervention. This is because these attributes are the “value add” of a blockchain-based intervention. For example, traditional financial inclusion interventions may seek to extend financial services of a bank to rural areas through digital money, extension agents, etc. A blockchain-based solution, however, may cut out the bank entirely and empower local communities to receive financial services from completely new providers from anywhere in the world on much more affordable terms in and in a much more convenient manner. Such a solution could see an alignment of interests amongst producers and consumers of these services since the new relationships are mutually serving. Because of this alignment there is a less of a need, or even less of a benefit, of having donors script out the causal pathway for the outcomes to be achieved. Because of this alignment of interests, those closest to the problem(s) and solutions can work it out because it is in their interest to do so.
Hence while a MEL framework for such a project could still use more standardized measures around outcomes like increased access to financial services and could even use statistical methods to evaluate questions around attributable changes in poverty status; there will need to be adaptive and formative MEL that assess the dynamics of these attributes given their criticality to whether and how outcomes could be achieved. The dynamics between these attributes and the surrounding social eco-system have the potential to be very fluid (going back to the disruptive nature of blockchain technology), hence flexible MEL will be required to respond to new trends as they emerge.
Table: Blockchain Intervention Attributes and the Skill Sets to Assess Them
| Blockchain Attributes | Possible MEL Approaches |
| Empowerment of those closest to problems to inform the relevant solutions | Problem driven design and MEL approach, stakeholder mapping (to identify relevant actors) Decentralization focused MEL (MEL that focuses on outcomes associated with decentralization) |
| Alignment of interests | Political economy analysis to identify incentives and interests Adaptive MEL to assess shifting alignment of interest between various actors |
| Alleviation of traditional intermediary services | Political economy analysis to inform risk mitigation strategy for potential spoilers and relevant MEL |
While there will need to be standard accountability and other uses, feedback from an appropriate MEL strategy could have two primary end uses in a blockchain based intervention: governance and trust.
The Role of Governance and Trust
Blockchain governance sets outs the rules for how consensus (ie. agreement) is achieved for deciding what transactions are valid on a blockchain. While this may sound mundane it is critical for achieving outcomes since how the blockchain is governed decides how well those closest to the problems are empowered to identify and achieve solutions and aligned interests. Hence the governance framework for the blockchain will need to be informed by an appropriate MEL strategy. A giant learning gap we currently have is how to iteratively adapt blockchain governance structures, using MEL feedback, into increasingly more efficient versions. Closing this gap will be critical to assessing the cost effectiveness of blockchain based solutions over other solutions (ie. alternatives/cost benefit analysis tools) as well as maximizing impact.
A giant learning gap we currently have is how to iteratively adapt blockchain governance structures, using MEL feedback, into increasingly more efficient versions.
Another focus of an appropriate MEL strategy would be to facilitate trust in the blockchain-based solution amongst users much the same as other technology-led solutions like mobile money or pay as you go metering for service delivery. This includes not only the digital interface between the user and the technology (a phone app, SMS or other interface) but other dimensions of “trust” that would facilitate uptake of the technology. These dimensions of trust would be informed by an analysis of the barriers to uptake of the technology amongst intended users, given it could be an entirely new service for beneficiaries or an old service delivered in a new fashion. There is already a good evidence base around what works in this area (ie. marketing and communication tools for digital financial services, assistance in completing registration paperwork for pay as you go metering, etc.).
The Road Ahead
There is A LOT we need to learn and a short time to do it in before we feel the negative effects from a lack of preparedness. This risk is heightened when you consider that the international development industry has a poor track record of designing and evaluating technology-led solutions (primarily due to the fact that these projects usually neglect uptake of the technology and operate on the assumption that the technology will drive outcomes instead of users using the technology as a tool to drive the outcomes).
The lessons from MEL in results based financing could be especially informative to the future of evaluating blockchain-based solutions given their similarities in letting solutions work themselves out and the role of the “validator” in ensuring outcomes are achieved. In fact the blockchain has already been used in this role in some simple output based programming.
As alluded to, pre-existing MEL skill sets can add a lot of value to building an evidence base but MEL practitioners will need to develop a greater understanding of the attributes of blockchain technology, otherwise our MEL strategies will not be suited to blockchain based programming.