VoteAmerica June 2020 ballot request program

Prepared by: Rachael Firestone, M.A. and Christopher Mann, Ph.D. with Annie Wang, J.D. Published: September 23, 2020

Summary: In a large-scale randomized controlled test during the June 2020 primaries in three states, VoteAmerica demonstrated that treatments using SMS messaging generate significant increases in mail ballot requests and subsequent ballot returns. Message testing indicates that message strategies emphasizing the COVID-19 pandemic were most effective at increasing ballot requests (+0.6 percentage points) and subsequent returns (+0.3 percentage points), while non-pandemic related messages also generated statistically significant but smaller impacts on ballot requests (+0.4 percentage points) and subsequent ballot returns (+0.1 percentage points). Importantly, the test in Georgia demonstrated coupling SMS treatments with VoteAmerica’s proprietary e-sign tool can increase mail ballot requests even in the absence of a state online ballot request website.

Background: Due to the challenges of voting in person during the COVID-19 pandemic, voting by mail is seen as a safe way for individuals to vote and an effective way to increase turnout. Within this generally under-studied method of voter mobilization, increasing turnout via SMS messages has received scant attention but a preliminary study in 2018 suggested it could be effective. The goal of this program was to inform future programs by establishing that SMS messages to encourage requesting mail ballots could increase turnout and shed light on which messages are most effective.

This program was conducted in three states during the June 2020 primaries. Two states had online ballot request websites (New Mexico and Pennsylvania). Georgia did not have an online ballot request website for the June primary, so ballot requests were facilitated using VoteAmerica’s proprietary e-sign tool that uploads and embeds a photograph of the voter’s signature on the voter registration form that is delivered to election officials by fax or email. (Georgia launched an online ballot request website in time for the November 2020 general election.)

Targets: 464,515 voters of color with high quality cell-phone numbers: Georgia (291,437), New Mexico (44,936), and Pennsylvania (128,142).

Randomized Controlled Tests: Records were randomly assigned to receive no contact from VoteAmerica or one of the treatments.

Random assignment provides clear, causal evidence of the effect of the treatment delivered by VoteAmerica. This research design, also known as field experiments, is considered the gold standard for voter mobilization evaluation by academic researchers and the Analyst Institute.

Treatment: The treatment consisted of two waves of text messages encouraging requesting a mail ballot. The text messages were delivered using MMS technology (the default for group text messaging on phones) because MMS allows the delivery of images as well as text. Records were randomly assigned to receive either a plain text MMS or an MMS with an image. The figures below show the images included in Waves 1 and 2.

Message Test: (see Appendix for copies of each message)

Friends & Family: This treatment successfully increased turnout in large-scale SMS voter mobilization programs in 2018. The key mechanism is a reminder that voting is a social activity, and also indirectly generate social pressure by reminding them that others (friends & family) will monitor this pro-social behavior.

Anti-coronavirus: This treatment was developed specifically for the pandemic and successfully increased turnout in earlier voter mobilization pilot tests in special elections. Motivation to return the ballot is increased by framing voting by mail as an action against the coronavirus and/or the uncertainty created by the pandemic.

Healthy & Safe: This treatment was developed specifically for the pandemic in qualitative research and survey experiments. It emphasizes the personal and public healthy benefits of voting by mail.

Anticipatory Benefits: This treatment was drawn from past research on encouraging early in- person voting. It is based on well-developed social psychology research about increased motivation to complete an action when it facilitates looking forward to a future event (e.g. making reservations for a vacation, purchasing tickets to a concert or sporting event).

Random Assignment:

The randomization used blocking to increase the probability of pre-treatment balance between randomly assigned conditions for voter characteristics that are correlated with voting behavior. Key variables used for blocking include prior experience voting by mail, prior experience voting, and race. Comprehensive blocking list: state, district, number of people of color in household, turnout score, race, number of cell phones in target universe in household, voted by mail in 2018, voted by mail in 2016, voted in 2018, voted in 2016

Timing: All Wave 1 messages were sent on May 18, 2020. Based on state deadlines to request mail ballots, Wave 2 messages were sent on May 23, 2020 (Pennsylvania), May 26, 2020 (New Mexico), and May 29 (Georgia).

Data and Analysis:

Ballot Requests

Message testing shows Anti-Coronavirus and Healthy and Safe messages generated larger increases in ballot requests than the alternative messages. The Anti-Coronavirus message caused a 0.63 percentage point increase in ballot requests (p<0.001). The Healthy and Safe message caused a 0.61 percentage point increase in ballot requests (p<0.001). The difference was not statistically significant between the Anti-Coronavirus and Healthy and Safe messages (p=0.89).

The Friends and Family and Anticipatory Benefits messages had smaller effects. Friends and Family caused a 0.32 percentage point increase in ballot requests (p=0.003) and Anticipatory Benefits caused a 0.40 percentage point increase in ballot requests (p=0.003). A full table of results is included in the appendix. The difference was also not statistically significant between Friends and Family and Anticipatory Benefits messages (p=0.56).

However, there was a statistically significant difference between each message with a larger effect and each message with a smaller effect. A full table of results from pairwise comparisons is included in the appendix.

There were no statistically significant differences between sending texts via MMS versus MMS with an image (p=0.64). Sending messages via MMS caused a 0.54 percentage point increase in ballot requests (p<0.001), and sending via MMS with an image caused a 0.49 percentage point increase (p<0.001).

Ballot Returns

(NB: VoteAmerica did not do any ballot chase program to the targets of the mail ballot request program.)

The same pattern of results for the messages holds when looking at the effect on ballots returned. For each treatment, slightly less than half of the net increase in ballot requests appeared as net increase in ballot returns in these low salience, generally uncompetitive primaries.

Anti-Coronavirus caused a 0.28 percentage point increase in ballots returned. Healthy and Safe caused a 0.29 percentage point increase in ballots returned. These effects for Anti-Coronavirus and Healthy and Safe messages were statistically significant at the 90% confidence level for ballot return results.

Friends and Family caused a 0.14 percentage point increase in ballots returned. Anticipatory Benefits caused a 0.08 percentage point increase in ballots returned. However, these results were not statistically significant. Full tables of results are included in the appendix.

Georgia: Effectiveness of VoteAmerica’s e-sign tool

In Georgia, individuals were directed to VoteAmerica’s proprietary e-sign tool to request a mail ballot. In Georgia, Anti-Coronavirus caused a 0.72 percentage point increase in ballot requests, Healthy and Safe caused a 0.62 percentage points in ballot requests, Friends and Family caused a 0.56 percentage point increase in ballot requests, and Anticipatory Benefits caused a 0.43 percentage point increase in ballot requests.

In Georgia, the effects on ballot returns were also statistically significant and substantively large: Anti-Coronavirus caused a 0.28 percentage point increase in ballot requests, Healthy and Safe caused a 0.26 percentage points in ballot returns, Friends and Family caused a 0.23 percentage point increase in ballot returns, and Anticipatory Benefits caused a 0.18 percentage point increase in ballot returns.

Although caution is needed when comparing rates across states, the higher than average effects in Georgia clearly indicates that VoteAmerica’s e-sign tool worked very effectively in facilitating ballot requests. A full table of results is included in the appendix.

Future Steps: These results suggest considerable promise for SMS based mail ballot recruitment efforts in 2020, especially when using pandemic oriented messaging. The change in requests rates was substantively large relative to other impacts on voting behavior from SMS messages. The increase in return rates was smaller, but still highly cost effective. Moreover, the conversion of ballot requests is likely to be higher in general elections than these low salience, generally uncompetitive primaries.

In particular, VoteAmerica’s e-sign tool demonstrated that it can effectively facilitate online ballot requests in the absence of online ballot request tools.

The negligible impact of adding images is surprising. Different images might have a larger positive impact or the impact may be seen in particular subgroups.

Future research should focus on replication and expanding message testing. While several messages were tested in this program, there may be other effective messages beyond the scope of this program and some may prove more effective than others.

Appendix