Can SMS-based Information Campaigns Help Prevent the Spread of COVID-19 in India?

Preventive behaviors such as handwashing and social distancing are critical to containing the spread of infectious diseases like COVID-19, particularly in densely populated areas of low-income countries with crowded living quarters and public spaces. But it can be challenging to identify and deploy effective interventions to encourage widespread knowledge and adoption of these behaviors.

Nudging from a distance

Information campaigns that use text messages (SMS) are a particularly appealing approach. Basic cell phones are ubiquitous and sharing information by phone does not require in-person contact, a key safety issue during a pandemic. Approaches using SMS are also inexpensive and scalable.

But do SMS information campaigns work, and what is the most effective design? We examined these questions in a recently completed a trial in Bihar, India, in which we tried to encourage social distancing and handwashing. In this blog, we summarize our working paper and contrast our findings with those of related studies in India that have used video messages or a combination of interactive voice calls and SMS.

Our randomized controlled trial of a SMS campaign in India

We worked with the nongovernmental organization Suvita to send SMS to encourage households in Bihar to adopt social distancing and hand washing. As part of its work with the government on childhood immunization, Suvita had collected phone numbers from birth registers at health centers in one district. We sent SMS to a random sample of households during the first peak of India’s COVID-19 pandemic between mid-August and mid-October 2020.

We were interested in the effect of SMS on the two health behaviors but also tried to identify the most effective design for this information campaign. There is ample evidence that how information is conveyed matters, for instance that a specific framing makes content more salient or better encourages prosocial behavior. Similarly, the timing of message delivery can impact the efficacy of an information campaign.

For these reasons, we assessed 10 approaches that differed in the message framing and time-of-day when they were sent. For the framing, we used five different versions that highlight gains or losses for either one’s own family or community, in addition to a neutral phrasing. We also varied the time of day when the messages were sent, either twice in the morning or once in the morning and once in the evening.

To test this many arms, we used an adaptive trial approach that—over the course of 10 rounds—identifies the combination of timing and message framing that is most effective. We conducted phone surveys with recipients three days after the first message was sent, along with surveys of control households who did not receive any messages.

One risk in surveys like this is social desirability bias: when prompted, respondents might say they adopted some socially acceptable behavior, even if they could not or chose not to actually adopt it. We used a range of elicitation methods to minimize this risk, including open-ended questions, questions about behaviors in the community, and a list experiment. In total we completed surveys with about 4,000 households.

Texts don’t nudge

Alas, we found no evidence that any of our SMS-based information campaigns improved knowledge or adoption of social distancing and handwashing. The main findings are summarized in Figure 1.

We cannot, however, reject potentially meaningful treatment effects. Looking at all of the study’s different treatment arms together for each target behavior, we cannot reject medium-sized impacts (about 5.5 percentage points or more) for adopting social distancing and handwashing. To put that differently: there could have been smaller impacts that could be meaningful but that we cannot detect in our study. We also find no evidence that messages focusing specifically on handwashing had an effect on social distancing behaviors (or vice versa), nor evidence that the impacts might vary over the span of our study period or by self-reported literacy, although our estimates for such impacts are imprecise. Our treatment group is more likely to report having received COVID-related messages, so we know the intervention was implemented relatively successfully—it just did not ultimately change behaviors.

There are several potential explanations for the lack of impacts in our study. Our study took place several months into the pandemic, at a time when cases were spiking and after households across India had received messages about COVID-19 mitigation and endured lockdowns for several months. Citizens might already be well-informed or too fatigued to respond to the nudges, something we also heard in a small number of qualitative interviews. Finally, our intervention featured relatively plain SMS messages, rather than more engaging content.

Star power or a personal touch?

That brings us to the handful of other studies on phone-based information campaigns in India that used different approaches.

A study by Abhijit Banerjee and colleagues sent video links to households in West Bengal in May 2020. The video features Prof. Banerjee—a West Bengal native and recent Nobel laureate—appealing to viewers to practice preventive health behaviors (the videos vary the behaviors and framing). They document positive impacts on social distancing, handwashing, and hygiene behaviors, as well as possible spillovers on non-recipients in the same community.

In another study in Uttar Pradesh and Bangladesh in April 2020, Abu Siddique and coauthors find that phone calls and phone calls paired with SMS improve both awareness and behavior of hand washing and social distancing, relative to merely sending SMS. The results from these studies are not directly comparable to ours, due to different contexts and timings, and also because of differences in how the outcomes are measured.

Takeaways for practice and research

Overall, we suggest caution with SMS-based information campaigns for health behavior change during the COVID-19 pandemic. At least several months into a pandemic, a basic SMS may not be sufficiently engaging to generate large behavioral responses, though they can be effective in other contexts and applications, for instance for routine childhood immunizations.  Other approaches to COVID mitigation might be more impactful but also more challenging as, e.g., celebrity videos may get more responses (but require smartphones) while interactive phone calls may be more engaging (but are more costly).

Finally, it would be valuable to have a “living” systematic review of studies on information campaigns and other types of interventions to encourage protective health behaviors, like this effort for drug treatments for COVID-19 but for intervention studies instead. That would help policymakers quickly compare across interventions and identify those may be effective in their setting. Such a resource could also make sense of the varying ways that study outcomes are defined and measured while promoting common approaches for use in future studies.