Tag Archives: SMS

Tech Is Easy, People Are Hard: Behavioral Design Considerations to Improve Mobile Engagement

By Cathy Richards

Mobile platforms are often a go-to when it comes to monitoring and evaluation in developing communities and markets. One provider of these platforms, Echo Mobile, is often asked, “what sort of response rate can I expect for my SMS survey?” or, “what percentage of my audience will engage my IVR initiative?” In this session at MERL Tech DC in September, Boris Maguire, CEO of Echo Mobile, walked participants through various case studies which highlight that the answer to that question largely depends on the project’s individual context and that there is ultimately no one size fits all solution. 

Echo Mobile is a platform that allows users to have powerful conversations over SMS, voice, and USSD for purposes such as monitoring and evaluation, field reporting, feedback, information access, market research and customer service. The platform’s user segments include consumer goods (20%), education and health (16%), M&E/Research (15%), agriculture and conservation (14%), finance and consulting (13%) and media and advocacy (7%). Its user types are primarily business (35%), non-profit (31%) and social enterprises (29%). 

The team at Echo Mobile has learned that regardless of the chosen mobile engagement technology, achieving MERL goals often rests on the design and psychology behind the mobile engagement strategy – the content, tone, language, and timing of communications and the underlying incentives of the audience. More often than not, the most difficult parts in mobile engagement are the human aspects (psychological, emotional, strategic) rather than the technological implementation. 

Because of this, Echo Mobile chose to dive deeper into the factors they believed influenced mobile engagement the most. Some of their beliefs included:

  • Responder characteristics: Who are you trying to engage with? It’s important to figure out who you are engaging with and tailor your strategy to them.
  • Social capital and trust: Do these responders have a reason to trust you? What is the nature of your relationship with them?
  • Style, tone & content: What specific words are you using to engage with them? Are you showing that you want to know more and that you care about them?
  • Convenience: What is the level of effort, time and resources that responders have to invest in order to engage with your organization?
  • Incentives/relevance: Do they have a reason to engage with your organization? Do they think you’ll understand them better? Will they get more of what they need?

Through informal analysis, Echo Mobile found that the factors most highly correlated with high rates of engagement are the time of day in which recipients receive the messaging, followed by reminders to engage. Financial incentives were found to be the least effective. However, case studies prove that context ultimately adds the most important component of the mobile engagement strategy.

In the first case study, a BBOXX team in Rwanda sought to understand the welfare impact of solar consumption amongst their customers via SMS surveys. They first ran a set of small experiments, modifying survey financial incentives, timing, length, and language to see which moved the needle on response rates and compare the results to what customers told them in focus groups. In this case, Echo Mobile found that reminders in the morning and surveys in the evening nearly doubled their response rates. The choice to opt or dive in also affected response rates.

In the second case study, a UN agency nearly doubled SMS engagement rates from 40,000 Kenyan teachers by dropping financial incentives and tweaking the structure, tone and content of their messaging. In this case, incentive amounts once again did not do much to increase engagement but rather the ability to opt or dive in, reminders, and content/tone made the biggest difference. 

In short, Echo Mobile’s biggest takeaways are that:

  • Convenience is king
  • One can harass but not bore
  • Financial incentives are often overrated

Several participants also shared their experiences with mobile engagement and cited factors such as survey length and consent as important. 

Tips for Increasing Mobile Survey Response Rates

Guest post from James Mwangi, the Deployment Lead at Echo Mobile. This post was originally published on the Echo Mobile blog.

Users often ask us, “what response rate will I get from my survey?”, or “how can I increase my survey’s response rate?”

The truth is …. it depends!

Response rates depend on your organisation, your respondents, and their motivation for responding. Most of our users assume that financial incentives are the most effective for stimulating engagement, and indeed research shows they can enhance response rates. But they are not always necessary and rarely sufficient. The design of your survey — its structure, tone and content — is equally important and often ignored.

In a recent SMS survey conducted for the third time on behalf of a UN agency and government ministry, Echo’s Deployment team demonstrated that minor adjustments to survey design can drastically increase response rates, regardless of financial incentives.

In May 2017, the team sent a survey with a KES 35 airtime incentive to 25,000 Kenyan government employees, 21% of whom completed it. In October 2017, Deployment sent the same survey to the same group with the same airtime incentive. This time only 16% completed it. In February 2018, we sent the survey again, with minor design tweaks and no financial incentives. The completion rate nearly doubled to 29%.

Win-win! Our client saved money by dropping the airtime transfers and got more results. More of their beneficiaries were able to engage and provide critical feedback. Here are the design changes we made to the survey. Consider them next time you’re using Echo for Monitoring and Evaluation (M&E):

Personalize the content

The Echo Platform allows users to personalize messages using standard fields — basic, common data points like name, ID and location, which can be stored in Echo contact profiles and can be integrated into large-scale messages.

Unlike in 2017, in the 2018 version of the UN survey, our Deployment team added the NAME field to the first SMS. As a result, all recipients immediately saw their name before automatically progressing to the first question. This builds a sense of trust, captures recipients’ attention, and is less likely to be mistaken for spam.

And you don’t need to to just stick to standard fields! Any prior response to a survey can be stored as a custom field. If you ask recipients their favorite football team and store the response as a custom field, the next time you send them SMS you can personalize your content even further: “Hi [NAME]. Hope [FOOTBALL_TEAM] is doing well this week….”

Skip the “opt-in”

The Echo platform’s survey builder allows you to add an invitation message as the first SMS sent to a contact. To move from this intro message on to the first question, recipients must “opt-in” by responding to this initial message with something like “ok” or “begin” (any word/number will do).

Sample survey designs, before optimisation.

Invitation messages are extremely useful. They help you be polite, introduce yourself if the recipient doesn’t know you, and say what your survey is about and why and how they can proceed (more below on instructions!). But they can also create a barrier to completion.

Observing that many respondents had failed to opt in to our 2017 survey, for the 2018 version of the survey we dropped the invitation message. Instead, we took that content and sent it as an info question, which, by design, automatically progress to the next question, regardless of a response or not.

Optimised survey ; personalised, does not require the respondent to opt in, and has clear instructions on how to reply.

Removing the opt-in invitation message won’t always be an option, but in this case, respondents were employees of our client and had been engaging on their shortcode for years. In some ways the intro message just added an extra step for them, as they had already provided their phone numbers and given consent to allow our client to engage them. Personally Identifiable Information (PII) is also not collected, nor shared and the respondents have an option to unsubscribe entirely from our system by sending the word STOP at any time, an option that has been communicated to them repeatedly.

In other cases, users might be suspicious of the opt-in request. Many Kenyans have encountered premium SMS services that push messages to unknowing respondents and deduct airtime from them once they opt in. Messaging with Echo is totally free for your respondents, but consider how they might react to an opt-in intro message, and design your survey accordingly!

Give clear Instructions

Keeping in mind SMS character limit, our Deployment team added quick instructions at the end of each question in the 2018 survey. These guided the respondents on how to answer specific question types. In the prior 2017 versions, each SMS had only contained the question, without instructions on how to answer:

Send reminders

For the 2017 surveys, we automated a reminder, sent 24 hours after the survey to those who had not yet started or completed it. For the 2018 version we added a second reminder, sent 12 hours later.

Reminders like these nudge contacts who are willing to respond to the survey but may have become distracted before completing it. This is especially true for long surveys like the one we have been deploying for the UN, which risk respondent fatigue. Reminders are a subtle way of urging them to finish the survey. Better yet — keep it lean!

So, what’s the take away here?

While research on the potential impact of financial incentives is clear, no amount of money or airtime can make up for suboptimal survey design!

Monetary rewards can move the response rate in the margins, but not always, and only if you get the design right first. Financial incentives are complementary to a well designed survey that has useful and clear content, an efficient structure, and a personal tone.

That said, non-financial incentives — the broader reasons why your contacts might want to engage with you at all — are an extremely important consideration. Not everyone’s time and information can be bought.

Consider for your next survey or engagement what informational, relational, or emotional incentives you might be explicitly or implicitly offering up front. As with any relationship, both sides ultimately need to feel like there is some benefit to the commitment. We’ll blog more about this idea soon!

Want to learn more from the Echo Deployment team? We consult on mobile engagement strategy and techniques, and can provide implementation support for survey creation, setup, optimization, deployment, and tracking on the Echo Platform.