VoteAmerica’s November 2020 ballot request program

Research and Analysis by Rachael Firestone, Jennifer Lauv, and Christopher Mann Memo and Additional Analysis by Scott Minkoff

For the 2020 general election cycle, VoteAmerica ran a 10-state text messaging (Peer-to-Peer SMS) campaign aimed at geng potential voters to request ballots and vote by mail. The ten states were: Alabama, Florida, Georgia, Kansas, Maine, Michigan, Mississippi, Montana, North Carolina, and Wisconsin.

In total, VoteAmerica contacted 5,532,915 potential voters with up to four different messages. Those four text-messages are presented in Figure 1. All of these potential voters were in VoteAmerica’s target universe of historically low-turnout populations including people of color, people under 35, and unmarried women.

For the later phases of the campaign, VoteAmerica sought to test the effectiveness of the text messages using an experiment: Were the messages causing recipients to request ballots and vote? To accomplish this, a limited number of recipients were randomly removed from the list to create a control group that could be compared to those who did receive the final text message.

To test the effect of treatment, ordinary least squares (OLS) with state-congressional district fixed effects was used. The models include an array of control variables: age, sex, marital status, race, TargetSmart Turnout Score, and voter history (turnout and use of absentee balloting).

The results – presented in Tables 1 and 2 – offer something of a mixed bag. When all targeted states are included in the model (Table 1), there is a positive effect for requested ballots that is significant at the 90% level but not a significant effect for actual voting. This means that people that received the fourth text message were slightly more likely to request a ballot.

However, when Georgia is excluded from the analysis (owing to the extreme political intensity of campaigns there in 2020, even relative to other competitive states) (Table 2), there is a notable effect for both ballot request and actual voting, again the 90% level. The effect amounted to just shy of half a percentage point for requesting a ballot and about seven-tenths of a percentage point for voter turnout. The identification of this effect among mid-propensity voters is important because research has consistently shown that these are the potential voters most likely to be impacted by mobilization campaigns.3 These results are visualized in Figure 2 where the red bars show the effect size, and the black whiskers show a 90% confidence interval (we can be 90% confident that the “true” effect lies within the whisker bar). The effect sizes are around what would be expected from past mobilization programs even if our confidence in them does not reach ideal thresholds.

It is essential to consider three points of context when evaluating VoteAmerica’s 2020 ballot request text-messaging program.

Point of Context 1: The 2020 election had historically high turnout and a campaign environment where political messaging was ubiquitous. Indeed, VoteAmerica’s text messaging program was run amongst an untold number of other text-message voter mobilization programs. This is not the ideal environment off which to gauge programmatic effectiveness for this kind of mobilization campaign because so many people were being inundated with similar political information.

Point of Context 2: The experiment was limited to the final phase of messaging. Had there been a more pure control group that received zero text messages, it is plausible that more substantial effects would have been observed.

Point of Context 3: In another effort to maximize the reach of the program, a very small control group was used: 1.14% (63,899) of the 5,596,814 people in the original target universe. While this control group was demographically similar to the group who received the last phase of text messages, its small relative size is not ideal for measuring effects.

The overall constellation of results is consistent with “effect attenuation.” That is, the intense election environment and the research design that that political environment necessitated both reduced the actual effect of the messaging and VoteAmerica’s ability to accurately observe that effect. That we observe some effects consistent with expectations is a good sign that when messages are appropriately targeted and the right political environments present themselves, the text-messages can be helpful.