Congress passed the National Voter Registration Act (NVRA) in 1993 in order to increase the number of eligible citizens who register to vote in federal elections. To help meet this goal, Section 7 of the NVRA requires state public assistance agencies to provide voter registration services to applicants and clients. Recent research has indicated that the number of voter registration applications from public assistance agencies has de- clined 79 percent since initial implementation of the law in 1995. In this paper, we examine the argument that the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) is an important cause of the decline in the number of individuals who have registered to vote in public assistance agencies since 1995. This argument was made most recently in an empirical analysis by Muhlhausen and Tyrrell in a report from the Heritage Center for Data Analysis (2008) titled “Welfare Reform a Factor in Lower Voter Registration at Public Assistance Offices.”

We conclude that the Muhlhausen and Tyrrell research suffers from methodological flaws that undermine the validity of their claims. Muhlhausen and Tyrrell make a key error of inference in overstating the certainty of their results. By artificially manipulating the number of observations included in their analysis, Muhlhausen and Tyrrell greatly overstate the precision of their estimates, thus allowing them to claim that their results reach traditional levels of statistical reliability when, in fact, they largely do not. In addition, our analysis indicates they make several other problematic methodological decisions that further call into question the validity of their results. 

Introduction

In 1993, the National Voter Registration Act (NVRA) was passed and it mandated that most states allow eligible individuals the opportunity to register to vote at government facilities, including government facilities that offer public assistance. But, as has been noted in a variety of recent studies, since implementation of the NVRA after 1995, the number of voter registrations occurring at public assistance agencies has fallen. The most recent of these studies found a decline of 79 percent in the number of public assistance registrations between 1995-1995 and 2005-2006.1 This has led to some debate regarding the causes of the decline in the number of voter registrations originating from public assistance agencies. Some have argued that the decline in public assistance voter registrations is due to a failure on the part of states to provide appropriate means for eligible individuals to have access to voter registration materials in public assistance agencies. Others have argued that the decline in public assistance voter registrations is due to changes in the availability of public assistance programs, arising from welfare reform legislation that was passed at about the same time that NVRA was originally being implemented in most states.

In this report we do not resolve this debate. Here we focus on the argument that the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA), passed in 1996, is an important cause of the decline in the number of individuals who have reg- istered to vote in public assistance agencies since 1995.

This argument was made most recently in an empirical analysis by Muhlhausen and Tyrrell, in a report from The Heritage Center for Data Analysis (2008) titled “Welfare Reform a Factor in Lower Voter Registration at Public Assistance Offices.” Their primary result comes from a statistical examination of data collected from most states on the number of public assistance voter registrations in those states since the first NVRA reporting period in 1995-1996, and running through the most recent NVRA reporting period of 2005-2006. Muhlhausen and Tyrrell use a statistical model, examining in particular the number of AFDC and TANF recipients in each state, while controlling for a variety of other public assistance programs’ usage, economic factors in each state, electoral variables, and state fixed effects. Much of Muhlhausen and Tyrrell’s analysis revolves around their claim to have found a generally statistically significant relationship between the number of AFDC and TANF recipients in a state, and the number of public assistance voter registrations in a state. Muhlhausen and Tyrrell state their primary result:

After controlling for other factors in Model 1, AFDC/TANF participation has a statistically significant association with public assistance registrations. A one-unit increase in AFDC/TANF participants per 100,000 residents is associ- ated with an increase of 0.062 additional registrations per 100,000 adult residents...A 1 percent increase in AFDC/TANF participation is associated with a 0.49 percent increase in voter registrations. Conversely, a 1 percent decrease in AFDC/TANF participation is associated with a 0.49 percent decline in voter registrations.

The first question we ask is what do their results imply for the overall level of public assis- tance voter registration applications over time. Assuming that Muhlhausen and Tyrrell’s results as reported for Model 1 in Table 2 of their report are correct (an assumption that we explain below is suspect), then controlling for the other effects in their model, we should see a drop of 62 public assistance voter registration applications per 100,000 VAP. But what does this imply for the important substantive question here, the overall levels of public assistance voter registration applications over this period?

We computed the net decrease in public assistance registrations from 1995 to 2006 that could be attributed to the decline in AFDC/TANF recipients based on Muhlhausen and Tyrrell’s reported estimates for their Model 1 for the states where the number of public assistance registrations was available for both years. According to Muhlhausen and Tyrrell’s estimated coefficient of .062, when translated into aggregate numbers, the drop in AFDC/TANF recipients would lead to a drop of 583,908 public assistance registrations.2 The actual drop in public assistance registration applications in these states from 1995 to 2006 was 729,836. We also computed the net decrease in public assistance registrations from 1995 to 2006 that could be attributed to declines in AFDC/TANF recipients, Food Stamp recipients, and WIC recipients based on Muhlhausen and Tyrrell’s estimates from Model 1. The Muhlhausen and Tyrrell estimates would predict a drop of 690,502 public assistance registrations over this time that could be attributed to the declining assistance rolls. 

Is The Muhlhausen And Tyrrell Estimate Believable?

Should we believe the Muhlhausen and Tyrrell estimate of the relationship between the numbers of AFDC/TANF recipients, Food Stamp recipients, WIC recipients, and the number of public assistance registration applications? Muhlhausen and Tyrrell make a key error of inference in overstating the certainty of their results. Furthermore, they make several statistical decisions that render their claims questionable.

Muhlhausen and Tyrrell report that their results are based on either 512 observations (1995-1996 to 2005-0206) or 424 observations (1997-98 to 2005-2006). Note that a “full’’ dataset for Muhlhausen and Tyrrell would include 540 observations: 44 states covered by the NVRA plus the District of Columbia for 12 years.3 However, in fact Muhlhausen and Tyrrell really only have data for half as many observations as they use in their analy- sis. The number of public assistance registration applications is only reported for each two-year election cycle. Muhlhausen and Tyrrell double their number of observations by arbitrarily dividing the number of public assistance registration applications by two, and assigning half the cycle’s total to each year. In other words, in half the observations that their analysis is based on, the key variable of interest is not really observed, but rather Muhlhausen and Tyrrell impute it in this crude way. This greatly overstates the precision of their estimates, allowing them to claim that their results reach traditional levels of statistical reliability, when in fact they largely do not. If we simply corrected for this double counting, then only one of the 12 estimated coefficients that Muhlhausen and Tyrrell report for the effects of assistance recipients on public assistance registrations would be statistically significant.4

Furthermore, advanced statistical techniques, such as the panel regression models used by Muhlhausen and Tyrrell require strong assumptions, and are best suited for applica- tions involving substantially larger datasets than the one they employ. Given the very short time period involved, Muhlhausen and Tyrrell are in fact trying to do panel analysis with data that only covers at best 5 or 6 election cycles.5 Using panel regression models with such small datasets might explain the glaring discrepancies in results across the estimates reported in Table 2, where Muhlhausen and Tyrrell report the estimates for four different model specifications. The models vary mainly in regards to whether they include data from the first NVRA reporting period (1995-1996), and in whether they include “Reporting year fixed effects”.6

Model 3 reports an estimated coefficient of -.009 for the most important estimate, the coefficient for AFDC/TANF recipients, while Model 4, estimated over the shorter pe- riod, reports an estimated coefficient of .062. In other words, one model predicts that an increase in AFDC/TANF recipients would cause a decrease in public assistance registrations, while the other model predicts that an increase in AFDC/TANF recipients would cause an increase in public assistance registrations!7 That the coefficient of the key independent variable can change so much based on inclusion of one additional time point, even in a model with period fixed effects, suggests that we should not be very confident in the estimates. Why not believe that the estimated coefficient is -.009, suggesting that an increase in AFDC/TANF recipients leads to a decrease in public assistance registrations? After all, this coefficient is estimated on more data than is the estimate of .062.

That their model estimates are so sensitive to these relatively minor changes in specifica- tion is likely due to the relatively small dataset that they use for their analysis; covering at best twelve years and missing a handful of states that did not report data or which are exempt from NVRA provisions. Thus, by not using the first NVRA reporting cycle (1995 and 1996), Muhlhausen and Tyrrell drop 88 cases from their analysis; nearly a fifth of their original dataset. It’s no wonder that with a fifth of the data suddenly missing that the results change so dramatically in their report!

Finally, we note that the number of states reporting public assistance registration data has declined over time. In the 2005-2006 cycle, 12 states did not report any public as- sistance registration data. Thus notice that if all states were reporting data, Muhlhausen and Tyrrell would be reporting 540 cases for their full period (44 NVRA-covered states, plus Washington DC, for 12 years). However, they actually report only 512 cases because of all the non-reporting of public assistance registration data by states. Thus any infer- ences drawn are based on an incomplete picture of the data. In the absence of knowing why some states were not reporting data, we cannot say whether or not this biases Muhlhausen and Tyrrell’s conclusions. 

Further Problems With Muhlhausen and Tyrrell's Model

We have tried to explain above that even assuming that the models that Muhlhausen and Tyrrell report are correct, the results are far less persuasive than they claim simply because of the limited sample size, and due to the artificial inflation of the sample used by Muhlhausen and Tyrrell. But furthermore, there are good reasons to believe that their statistical model is incorrectly specified. There are many other explanations for why otherwise eligible individuals might not register to vote in a public assistance office than the decline in AFDC/TANF recipients.

One of those reasons is what motivated the original Kavanagh et al. study—that some states may not implement voter registration procedures in public assistance agencies appropriately.8 Nowhere in their study do Muhlhausen and Tyrrell introduce any variables that measure changes in NVRA implementation over time, and thus their model does not test the competing hypothesis that the decline in public assistance agency registra- tions stems from some states’ inadequate implementation of NVRA.9 If states with in- creasing levels of AFDC/TANF recipients were also states that have been declining in their effective implementation of the NVRA, then we would observe exactly the rela- tionship Muhlhausen and Tyrrell do in Model 1—but it would not be caused by AFDC/ TANF levels, but by inadequate implementation of NVRA. There is ample evidence that implementation matters. When North Carolina voluntarily changed their NVRA procedures in response to complaints of non-compliance, the number of public assistance registrations per month increased over six-fold. Furthermore, when Missouri changed its implementation plan in response to a court-order, the number of public assistance registrations per month increased over twenty-five-fold.10

There are other missing causal variables, including some that oddly enough Muhlhausen and Tyrrell discuss but never seem to return to, for example: “Other possible explana- tions for the decline include voter registration drives by community mobilization orga- nizations...and welfare reform...” (page 2). Nowhere do they control for the rise in community mobilization efforts, another potentially competing causal effect that is omitted from their model and whose omission might lead to incorrect estimates and problematic inferences

Additionally, by focusing only on AFDC/TANF caseload in their model, Muhlhausen and Tyrrell have ignored what may be more appropriate measures of the number of in- dividuals who should have been offered voter registration. The NVRA requires voter registration services to be provided with each application, recertification, and change of address related to benefits, not simply to clients who successfully make it onto the assis- tance rolls and are thus included in caseload data. Indeed, there is evidence indicating that, at least in some states, applications for AFDC/TANF were actually increasing dur- ing the time period in which caseloads were declining.11

Finally, Muhlhausen and Tyrrell choose to weight their data by state population. This weighting is hard to justify in any sort of multivariate model. But to us it makes little sense in a multivariate model where the unit of analysis is the state. Muhlhausen and Tyrrell are giving more weight to large states, for no apparent reason. The precise question at hand is the effect of individual state implementation of the NVRA provisions. There is no more reason to weight states by population than there is to weight them by state geographic size. 

Conclusion

In the end, we do not know with certainty that Muhlhausen and Tyrrell are wrong that a decrease in AFDC/TANF rolls accounts for some of the decline in public assistance registrations. However, we do know that their estimates are much less precise than they claim. We also know that even by their own estimates, the decline in AFDC/TANF rolls do not account for the entire decline in public assistance registrations. Thus to suggest that states are all effectively implementing the NVRA, even while more states than ever are not reporting the data needed to verify implementation of NVRA, is too strong a conclusion to take from Muhlhausen and Tyrrell’s analysis of six election cycles. 

Endnotes

  1. Douglas R. Hess and Scott Novakowski, Unequal Access: Neglecting the National Voter Registration Act, 1995-2007, (February 2008), available at http://www.demos.org/publication/unequal-access-neglecting-national-voter-registration-act-1995-2007

  2. While these numbers are only officially reported for two year election cycles, we report annualized numbers throughout to maintain comparability with the manner in which Muhlhausen and Tyrrell handle the data, which we describe below.

  3. Idaho, Minnesota, New Hampshire, Wisconsin and Wyoming are exempt from the NVRA because they offered Election Day Registration at the polling place at the time the Act was passed. North Dakota is exempt because it has no statewide voter registration requirement.

  4. In the absence of any other issues, this correction would leave their coefficient estimates un- changed, but inflate their standard errors by a factor of 1.4. By failing to reach statistical significance, it indicates that there is a reasonable chance that Mulhausen and Tyrrell are finding ‘effects’ when in fact no effects really exist. We use the traditional level of statistical significance, 95%, throughout.

  5. Muhlhausen and Tyrrell use both the periods 1995-1996 to 2005-2006, and 1997-1998 to 2005- 2006.

  6. Muhlhausen and Tyrrell define “Reporting year fixed effects” as “individual time-period dummy variables...for the 1997-1998 to 2005-2006 period.” In statistical terms, in these models they are controlling for any secular shift to public assistances registration rates that happened in each year across all of the states.

  7. Both of these models include period fixed effects.

  8. Brian Kavanagh, Steven Carbo, Lucy Mayo and Michael Slater, Ten Years Later, A Promise Unfulfilled, (September 2005).

  9. It is well known by students of statistics that model misspecification, omitting a potentially com- peting causal variable from a multivariate statistical model, can lead to incorrect estimates and improper statistical inference.

  10. These changes in implementation are documented in Scott Novakowski, Towards an Equal Elector- ate: Five States’ Gains under the National Voter Registration Act, (Dēmos, 2008), available at http://www.demos.org/publication/toward-equal-electorate-five-states-gains-under-national-voter-registration-act

  11. For example, in New York State, the number of TANF recipients decreased by 15 percent between 2002 and 2007 while the number of applications for TANF increased by 22 percent. See http:// www.otda.state.ny.us/MAIN/reports/legislativereport.asp. See also Julie Bosman, “As More Apply for Welfare, Concern for Those Denied,” New York Times, p. A19, April 29, 2009, available at http:// www.nytimes.com/2009/04/29/nyregion/29welfare.html.