2017 Virginia cold SMS GOTV program evaluation

Program conception and design by Debra Cleaver (VoteAmerica) and Christopher Mann (Skidmore College). Evaluation by Michael Schwam-Baird, Jeff Ferguson, and Miya Woolfalk (Analyst Institute).

Executive Summary

In the lead up to the November 2017 gubernatorial election in Virginia, a national civic organization (NCO) partnered with Analyst Institute to evaluate a “cold” SMS GOTV program, an SMS program in which there is no pre-existing relationship between the sending organization and the receiver. The primary targets of the program were unmarried women and voters of color. The test was designed to evaluate whether receiving texts in four waves over a month-long period before the election was more effective than only receiving texts in the week before the election. The test also examined the effects of different SMS messages that provided polling place information, ballot information, or an encouragement to vote based on social norms.

Key findings:

• Individuals in the SMS program voted at a rate 0.6 percentage points (pp, p < 0.01) higher than the control group, generating approximately 2,570 net voters.This effect is at the high end of what we have observed; other “cold” SMS GOTV programs have boosted turnout by up to 0.2pp. Votes were generated at an efficient $31 per vote, or 32 votes per $1,000.
• Late texts alone were as effective as receiving all texts. Receiving text messages in the week right before the election boosted turnout by the same amount (0.6pp, p < 0.01) as receiving texts over a month-long period leading up to the election. Because the late text program was cheaper to implement, it produced votes at $11 per vote, or 88 votes per $1,000.
• The SMS message group that provided ballot information plus polling place location may have increased turnout relative to the message group that offered just polling place location. The treatment effect for targets who received a polling place message and a message with a link to look up their ballot was 0.7pp (p < 0.01). Targets who received the polling place message and a social norms message emphasizing that turnout was expected to be high voted at a rate 0.6pp (p < 0.01) higher than the control group. Finally, the treatment effect for targets who received just polling place reminders as their final text was 0.5pp (p < 0.01). The 0.2pp difference between the ballot information plus polling place location group and the polling place only group was marginally statistically significant (p = 0.15).This test produced important lessons about how to optimize GOTV SMS programs. The results demonstrate that sending GOTV text messages in an off-cycle election year can be an effective and cost efficient mobilization tactic. Moreover, sending text messages in the week before an election can be as effective as, and cheaper than, sending messages over a month-long period leading up to an election. Finally, while polling place location messages alone can increase turnout, programs that provide ballot information alongside polling place location information may generate further boosts to voter engagement, at least in lower-salience election environments where voters may have less information about items on the ballot. This suggestive finding provides a promising direction for future research.