Tag Archives: technology

We have a data problem

by Emily Tomkys, ICT in Programmes at Oxfam GB

Following my presentation at MERL Tech, I have realised that it’s not only Oxfam who have a data problem; many of us have a data problem. In the humanitarian and development space, we collect a lot of data – whether via mobile phone or a paper process, the amount of data each project generates is staggering. Some of this data goes into our MIS (Management Information Systems), but all too often data remains in Excel spreadsheets on computer hard drives, unconnected cloud storage systems or Access and bespoke databases.

(Watch Emily’s MERL Tech London Lightning Talk!)

This is an issue because the majority of our programme data is analysed in silos on a survey-to-survey basis and at best on a project-to-project basis. What about when we want to analyse data between projects, between countries, or even globally? It would currently take a lot of time and resources to bring data together in usable formats. Furthermore, issues of data security, limited support for country teams, data standards and the cost of systems or support mean there is a sustainability problem that is in many people’s interests to solve.

The demand from Oxfam’s country teams is high – one of the most common requests the ICT in Programme Team receive centres around databases and data analytics. Teams want to be able to store and analyse their data easily and safely; and there is growing demand for cross border analytics. Our humanitarian managers want to see statistics on the type of feedback we receive globally. Our livelihoods team wants to be able to monitor prices at markets on a national and regional scale. So this motivated us to look for a data solution but it’s something we know we can’t take on alone.

That’s why MERL Tech represented a great opportunity to check in with other peers about potential solutions and areas for collaboration. For now, our proposal is to design a data hub where no matter what the type of data (unstructured, semi-structured or structured) and no matter how we collect the data (mobile data collection tools or on paper), our data can integrate into a database. This isn’t about creating new tools – rather it’s about focusing on the interoperability and smooth transition between tools and storage options.  We plan to set this up so data can be pulled through into a reporting layer which may have a mixture of options for quantitative analysis, qualitative analysis and GIS mapping. We also know we need to give our micro-programme data a home and put everything in one place regardless of its source or format and make it easy to pull it through for analysis.

In this way we can explore data holistically, spot trends on a wider scale and really know more about our programmes and act accordingly. Not only should this reduce our cost of analysis, we will be able to analyse our data more efficiently and effectively. Moreover, taking a holistic view of the data life cycle will enable us to do data protection by design and it will be easier to support because the process and the tools being used will be streamlined. We know that one tool does not and cannot do everything we require when we work in such vast contexts, so a challenge will be how to streamline at the same time as factoring in contextual nuances.

Sounds easy, right? We will be starting to explore our options and working on the datahub in the coming months. MERL Tech was a great start to make connections, but we are keen to hear from others about how you are approaching “the data problem” and eager to set something up which can also be used by other actors. So please add your thoughts in the comments or get in touch if you have ideas!

Dropping down your ignorance ratio: Campaigns meet KNIME

by Rodrigo Barahona (Oxfam Intermon, @rbarahona77) and Enrique Rodriguez (Consultant, @datanauta)

A few year ago, we ran a Campaign targeting the Guatemalan Government, which generated a good deal of global public support (100,000 signatures, online activism, etc.). This, combined with other advocacy strategies, finally pushed change to happen. We did an evaluation in order to learn from such a success and found a key area where there was little to learn because we were unable to get and analyze the information:  we knew almost nothing about which online channels drove traffic to the online petition and which had better conversion rates. We didn’t know the source of more than 80% of our signatures, so we couldn’t establish recommendations for future similar actions

Building on the philosophy underneath Vanity Metrics, we started developing a system to evaluate public engagement as part of advocacy campaigns and spike actions. We wanted to improve our knowledge on what works and what doesn’t on mobilizing citizens to take action (mostly signing petitions or other online action), and which were the most effective channels in terms of generating traffic and converting petitions. So we started implementing a relatively simple Google Analytics Tracking system that helped us determine the source of the visit/signatures, establish conversion rates, etc. The only caveat was that it was time consuming — the extraction of the information and its analysis was mostly manual.

Later on, we were asked to implement the methodology on a complex campaign that had 3 landing/petition pages, 3 exit pages, and all this in two different languages. Our preliminary analysis was that it would take us up to 8-10 hours of work, with high risk of mistakes as it needed cross analysis of up to 12 pages, and required distinguishing among more than 15 different sources for each page.

But then we met KNIME: an Information Miner tool that helped us to extract different sets of data from Google analytics (through plugins), create the data flow in a visual way and automatically execute part of the analysis. So far, we have automated the capture and analysis of statistics of web traffic (Google Analytics), the community of users on Twitter and the relevance of posts in that social network. We’ve been able to minimize the risk of errors, focus on the definition of new indicators and visualizations and provide reports to draw conclusions and design new communication strategies (based on data) in a very short period of time.

KNIME helped us to scale up our evaluation system, making it suitable for very complex campaigns, with a significant reduction of time dedication and also lowering the risk of mistakes. And most important of all, introducing KNIME into our system has dropped down our ignorance ratio significantly, because nowadays we can identify the source of more than 95% of the signatures. This means that we can shed light on how different strategies are working, which channels are bringing more visits to the different landing pages, and which have the higher conversion rate. All this is relevant information to inform decisions and adapt strategies and improve the outputs of a campaign.

Watch Rodrigo’s MERL Tech Lightning Talk here!