Written by Jana Melpolder, MERL Tech DC Volunteer and former ICT Works Editor. Find Jana on Twitter: @JanaMelpolder
As organizations grow, they become increasingly aware of how important MERL (Monitoring, Evaluation, Research, and Learning) is to their international development programs. To meet this challenge, new hires need to be brought on board, but more importantly, changes need to happen in the organization’s culture.
How can nonprofits and organizations change to include more MERL? Friday afternoon’s MERL Tech DC session “Creating a MERL Culture at Your Nonprofit” set out to answer that question. Representatives from Salesforce.org and Samaschool.org were part of the discussion.
Salesforce.org staff members Eric Barela and Morgan Buras-Finlay emphasized that their organization has set aside resources (financial and otherwise) for international and external M&E. “A MERL culture is the foundation for the effective use of technology!” shared Eric Barela.
Data is a vital part of MERL, but those providing it to organizations often need to “hold the hands” of those on the receiving end. What is especially vital is helping people understand this data and gain deeper insight from it. It’s not just about the numbers – it’s about what is meant by those numbers and how people can learn and improve using the data.
According to Salesforce.org, an organization’s MERL culture is comprised of its understanding of the benefit of defining, measuring, understanding, and learning for social impact with rigor. And building or maintaining a MERL culture doesn’t just mean letting the data team do whatever they like or being the ones in charge. Instead, it’s vital to focus on outcomes. Salesforce.org discussed how its MERL staff prioritize keeping a foot in the door in many places and meeting often with people from different departments.
Where does technology fit into all of this? According to Salesforce.org, the push is on keep the technology ethical. Morgan Buras-Finlay described it well, saying “technology goes from building a useful tool to a tool that will actually be used.”
Another participant on Friday’s panel was Samaschool’s Director of Impact, Kosar Jahani. Samaschool describes itself as a San Francisco-based nonprofit focused on preparing low-income populations to succeed as independent workers. The organization has “brought together a passionate group of social entrepreneurs and educators who are reimagining workforce development for the 21st century.”
Samaschool creates a MERL culture through Learning Calls for their different audiences and funders. These Learning Calls are done regularly, they have a clear agenda, and sometimes they even happen openly on Facebook LIVE.
By ensuring a high level of transparency, Samasource is also aiming to create a culture of accountability where it can learn from failures as well as successes. By using social media, doors are opened and people have an easier time gaining access to information that otherwise would have been difficult to obtain.
Kosar explained a few negative aspects of this kind of transparency, saying that there is a risk to putting information in such a public place to view. It can lead to lost future investment. However, the organization feels this has helped build relationships and enhanced interactions.
Sadly, flight delays prevented a third organization. Big Elephant Studios and its founder Andrew Means from attending MERL Tech. Luckily, his slides were presented by Eric Barela. Andrew’s slides highlighted the following three things that are needed to create a MERL Culture:
Tools – investments in tools that help an organization acquire, access, and analyze the data it needs to make informed decisions
Processes – Investments in time to focus on utilizing data and supporting decision making
Culture – Organizational values that ensure that data is invested in, utilized, and listened to
One of Andrew’s main points was that generally, people really do want to gain insight and learn from data. The other members of the panel reiterated this as well.
A few lingering questions from the audience included:
How do you measure how culture is changing within an organization?
How does one determine if an organization’s culture is more focused on MERL that previously?
Which social media platforms and strategies can be used to create a MERL culture that provides transparency to clients, funders, and other stakeholders?
What about you? How do you create and measure the “MERL Culture” in your organization?
By Ben Mann, Policy & engineering nerd. Technology & data evangelist. Working for @DAIGlobal. The original appears here.
We live in the digital era. And the digital era is built on data. Everyone in your business, organization, agency, family, and friend group needs data. We don’t always realize it. Some won’t acknowledge it. But everyone needs and uses data every day to make decisions. One of my colleagues constantly reminds me that we are all data junkies who need that fix to “get sh*t done.”
So we all agree that we need data, right? Right.
Now comes the hard part: how do we actually use data? And not just to inform what we should buy on Amazon or who we should follow on Twitter, but how do we do the impossible (and over-used-buzzword of the century) to “make data-driven-decisions?” As I often hear from frustrated friends at conferences or over coffee, there is a collectively identified need for improving data literacy and, at the same time, collective angst over actually improving the who/what/where/when/why/how of data at our companies or organizations.
The short answer: We need to build our own data culture.
It needs to be inclusive and participatory for all levels of data users. It needs to leverage appropriate technology that is paired with responsible processes. It needs champions and data evangelists. It needs to be deep and wide and complex and welcoming where there are no stupid questions.
The long answer: We need to build our own data cultures. And it’s going to be hard. And expensive. And it’s an unreachable destination.
I was blessed to hear Shash Hegde (Microsoft Data Guru extraordinare) talk about modern data strategies for organizations. He lays out 6 core elements of a data strategy that any team needs to address to build a culture that is data-friendly and data-engaged:
Vision: Does your organization know their current state of data? Is there a vision for how it can be used and put to work?
People: Maybe more important than anything else on this list, people matter. They are the core of your user group, the ones who will generate most of your data, manage the systems, and consumer the insights. Do you know their habits, needs, and desires?
Structure: Not to be confused with stars or snowflakes — we mean the structure of your organization. How business units are formed, who manages what, who controls what resources, and how the pieces fit together.
Process: As a systems thinkings person, I know that there is always a process in play. Even the abscence of process is a process in and of itself. Knowing the process and workflow of your data is critical to flow and use of your culture.
Rules: They govern us. They set boundaries and guiding rails, defining our workspaces and playing fields.
Tools+Tech: We almost always start here, but I’d argue it is the least important. With the cloud and modern data platforms, with a sprinkling of AI and ML, it is rarely the bottleneck anymore. It’s important, but should never be the priority.
Building data culture is a journey. It can be endless. You may never achieve it. And unlike the Merry Pranksters, we need a destination to drive towards in building data literacy, use, and acceptance. And if anyone tells you that they can do it cheap or free, please show them the exit ASAP.
Starting your data adventure
At MERLTech DC, we recently hosted a panel on organizational data literacy and our desperate need for more of it. Experts (smarter than me) weighed in on how the heck we get ourselves, our teams, and our companies onto the path to data literacy and a data loving culture.
Three tangible things we agreed on:
💪🏼Be the champion.
Because someone has to, why not you?
👩🏾💼Get a senior sponsor.
Unless you are the CEO, you need someone with executive level weight behind you. Trust us (& learn from our own failures).
🧗🏽♂️Keep marching on. And invite everyone to join you.
You will face obstacles. You’ll face failures. You may feel like you’re alone. But helping lead organizational change is a rewarding experience — especially with something as awesome as data. It’s a journey everyone should be on and I encourage you to bring along as many coworkers/coconspirators/collaborators as possible. Preferably everyone.
So don’t wait any longer. Start your adventure in your organization today!!
by Maliha Khan, a development practitioner in the fields of design, measurement, evaluation and learning. Maliha led the Maturity Model sessions at MERL Tech DC and Linda Raftree, independent consultant and lead organizer of MERL Tech.
MERL Tech is a platform for discussion, learning and collaboration around the intersection of digital technology and Monitoring, Evaluation, Research, and Learning (MERL) in the humanitarian and international development fields. The MERL Tech network is multidisciplinary and includes researchers, evaluators, development practitioners, aid workers, technology developers, data analysts and data scientists, funders, and other key stakeholders.
One key goal of the MERL Tech conference and platform is to bring people from diverse backgrounds and practices together to learn from each other and to coalesce MERL Tech into a more cohesive field in its own right — a field that draws from the experiences and expertise of these various disciplines. MERL Tech tends to bring together six broad communities:
traditional M&E practitioners, who are interested in technology as a tool to help them do their work faster and better;
development practitioners, who are running ICT4D programs and beginning to pay more attention to the digital data produced by these tools and platforms;
business development and strategy leads in organizations who want to focus more on impact and keep their organizations up to speed with the field;
tech people who are interested in the application of newly developed digital tools, platforms and services to the field of development, but may lack knowledge of the context and nuance of that application
data people, who are focused on data analytics, big data, and predictive analytics, but similarly may lack a full grasp of the intricacies of the development field
donors and funders who are interested in technology, impact measurement, and innovation.
Since our first series of Technology Salons on ICT and M&E in 2012 and the first MERL Tech conference in 2014, the aim has been to create stronger bridges between these diverse groups and encourage the formation of a new field with an identity of its own — In other words, to move people beyond identifying as, say, an “evaluator who sometimes uses technology,” and towards identifying as a member of the MERL Tech space (or field or discipline) with a clearer understanding of how these various elements work together and play off one another and how they influence (and are influenced by) the shifts and changes happening in the wider ecosystem of international development.
By building and strengthening these divergent interests and disciplines into a field of their own, we hope that the community of practitioners can begin to better understand their own internal competencies and what they, as a unified field, offered to international development. This is a challenging prospect, as beyond their shared use of technology to gather, analyze, and store data and an interest in better understanding how, when, why, where, (etc.) these tools work for MERL and for development/humanitarian programming, there aren’t many similarities between participants.
At the MERL Tech London and MERL Tech DC conferences in 2017, we made a concerted effort to get to the next level in the process of creating a field. In London in February, participants created a timeline of technology and MERL and identified key areas that the MERL Tech community could work on strengthening (such as data privacy and security frameworks and more technological tools for qualitative MERL efforts). At MERL Tech DC, we began trying to understand what a ‘maturity model’ for MERL Tech might look like.
What do we mean by a ‘maturity model’?
Broadly, maturity models seek to qualitatively assess people/culture, processes/structures, and objects/technology to craft a predictive path that an organization, field, or discipline can take in its development and improvement.
Initially, we considered constructing a “straw” maturity model for MERL Tech and presenting it at the conference. The idea was that our straw model’s potential flaws would spark debate and discussion among participants. In the end, however, we decided against this approach because (a) we were worried that our straw model would unduly influence people’s opinions, and (b) we were not very confident in our own ability to construct a good maturity model.
Instead, we opted to facilitate a creative space over three sessions to encourage discussion on what a maturity model might look like, and what it might contain. Our vision for these sessions was to get participants to brainstorm in mixed groups containing different types of people- we didn’t want small subsets of participants to create models independently without the input of others.
In the first session, “Developing a MERL Tech Maturity Model”, we invited participants to consider what a maturity model might look like. Could we begin to imagine a graphic model that would enable self-evaluation and allow informed choices about how to best develop competencies, change and adjust processes and align structures in organizations to optimize using technology for MERL or indeed other parts of the development field?
In the second session, “Where do you sit on the Maturity Model?” we asked participants to use the ideas that emerged from our brainstorm in the first session to consider their own organizations and work, and compare them against potential maturity models. We encouraged participants to assess themselves using green (young sapling) to yellow (somewhere in the middle) and red (mature MERL Tech ninja!) and to strike up a conversation with other people in the breaks on why they chose that color.
In our third session, “Something old, something new”, we consolidated and synthesized the various concepts participants had engaged with throughout the conference. Everyone was encouraged to reflect on their own learning, lessons for their work, and what new ideas or techniques they may have picked up on and might use in the future.
The Maturity Models
As can be expected, when over 300 people take marker and crayons to paper, many a creative model emerges. We asked the participants to gallery walk the models over the next day during the breaks and vote on their favorite models.
We won’t go into detail of what all the 24 the models showed, but there were some common themes that emerged from the ones that got the most votes – almost all maturity models include dimensions (elements, components) and stages, and a depiction of potential progression from early stages to later stages across each dimension. They all also showed who the key stakeholders or players were, and some had some details on what might be expected of them at different stages of maturity.
Two of the models (MERLvana and the Data Appreciation Maturity Model – DAMM) depicted the notion that reaching maturity was never really possible and the process was an almost infinite loop. As the presenters explained MERLvana “it’s an impossible to reach the ideal state, but one must keep striving for it, in ever closer and tighter loops with fewer and fewer gains!”
“MERL-tropolis” had clearly defined categories (universal understanding, learning culture and awareness, common principles, and programmatic strategy) and the structures/ buildings needed for those (staff, funding, tools, standard operating procedures, skills).
The most popular was “The Data Turnpike” which showed the route from the start of “Implementation with no data” to the finish line of “Technology, capacity and interest in data and adaptive management” with all the pitfalls along the way (misuse, not timely, low ethics etc) marked to warn of the dangers.
As organizers of the session, we found the exercises both interesting and enlightening, and we hope they helped participants to begin thinking about their own MERL Tech practice in a more structured way. Participant feedback on the session was on polar extremes. Some people loved the exercise and felt that it allowed them to step back and think about how they and their organization were approaching MERL Tech and how they could move forward more systematically with building greater capacities and higher quality work. Some told us that they left with clear ideas on how they would work within their organizations to improve and enhance their MERL Tech practice, and that they had a better understanding of how to go about that. A few did not like that we had asked them to “sit around drawing pictures” and some others felt that the exercise was unclear and that we should have provided a model instead of asking people to create one. [Note: This is an ongoing challenge when bringing together so many types of participants from such diverse backgrounds and varied ways of thinking and approaching things!]
We’re curious if others have worked with “maturity models” and if they’ve been applied in this way or to the area of MERL Tech before. What do you think about the models we’ve shared? What is missing? How can we continue to think about this field and strengthen our theory and practice? What should we do at MERL Tech London in March 2018 and beyond to continue these conversations?