Cross-posted at Latino Decisions, Latino Vote Matters, and Daily Kos:
Did you know that, despite running exceedingly racialized anti-Latino advertisements in Nevada’s Senate race, Sharon Angle got 30% of the Latino vote? Jan Brewer also did well, attracting an above-average 28% share of Arizona’s Hispanic vote just months after signing SB1070 into law. Moreover, Meg Whitman received 21% of the African American vote in her failed run for Governor of California, including 28% of black males!
If these “facts” sound suspicious to you, they should. Nevertheless, they represent several of the formal findings of the National Exit Poll from 2010.
As we approach the election night for 2012, we again want to caution the press and the public. The National Exit Polls are very GOOD at calling overall state results in terms of the two-party vote and even better at getting the internals of the white, non-Hispanic vote. They do far less well in accurately capturing the vote of Latinos, other racial and ethnic minorities, and—indeed—any sub-group not evenly distributed throughout the US. And with the issue of language, the exit polls do a particularly bad job for Latinos. In January 2005 in the aftermath of the 2004 Bush-Kerry election, Warren Mitofsky, the head of the 2004 National Exit Poll, wrote in a self-assessment of the erroneous exit poll data, “Some estimates differ by several points among certain demographic groups, most noticeably among Hispanics. These differences appear mostly among demographic groups that are both relatively small and those that tend to be geographically concentrated.” He concluded that his own previous estimate of 44% of Latinos voting for Bush was wrong.
THREE QUESTIONS REPORTERS SHOULD RAISE WHEN PRESENTED WITH ANY EXIT POLL RESULTS ON LATINOS.
1) How were precincts selected? What criteria apart from “probability proportionate to size” were used? How were “national” precincts selected from the state samples? What did NEP do in the 19 states where there are NO state level studies?
The selection of precincts by the NEP is something of a mystery. While there was text on their website that asserted that all voters have an approximately equal opportunity of selection, it is clear that the resulting precincts are also influenced by geography and vote history. Indeed, their own description suggests precincts are grouped by “type,” which is not defined.
In recent years, “national” estimates were drawn from surveys administered in a set of precincts sampled from those already selected within each state. This year, 19 states have no state-level polls. How are precincts selected in these states? How many?
The selection of precincts matters a great deal. Because all the interviews are clustered in the selected precincts, the number of precincts in which surveys are conducted is critical. Too few precincts and the margin of error around estimates for sub-populations like Latinos or African Americans will be very large. And if there is a systematic bias—for example, if NEP selects a somewhat larger share of competitive precincts to better predict two-party vote—then we over-survey a special kind of minority voter, one who lives primarily among whites. These voters are more assimilated, better educated, higher income, and more conservative than other minority voters.
As we note just above, Warren Mitofsky—the pollster behind the formation of the exit polls as we now know them—admitted that clustering effects can particularly effect Latino data, “A detailed look at the distribution of plurality Hispanic precincts in the National Exit Poll sample demonstrates how this clustering effect can influence the estimate of Hispanic voting in the National Exit Poll.” He concludes, “If we want to improve the National Exit Poll estimate for Hispanic vote we would either need to drastically increase the number of precincts in the National Sample or oversample the number of Hispanic precincts,” (Mitofsky 2005).
A post-2004 reexamination (Leal et al 2005) of the exit polls whose Latino estimates diverged substantially from all pre-election polling found that precincts with significant minority majorities—which represent the demographic reality of many minority voters—are wildly underrepresented in the national frame.
2) What share of Latino interviews were done in Spanish? If that number is much less than 25-30%, how can your poll be accurate if between a quarter and a third of Latino voters are Spanish-dominant?
Pew estimates that 25% of Latino voters are foreign born. An additional group is comprised of natural born citizens of the US who were born on the island of Puerto Rico (or moved back and forth across their lifespan). An additional small percent are native-born citizens but have lived in linguistically isolated communities. As a consequence, somewhere higher than 30% of Latino registered voters nationally would prefer to be interviewed in Spanish (this varies by state).
When Latino Decisions polls registered voters at random, all respondents are offered an immediate opportunity to speak Spanish. Since our interviewers are bilingual, this does not require a call back or even a delay—the interviewer proceeds in Spanish without interruption. Exit polls provide no such opportunity. Spanish language interviewing is available only on a limited basis in majority-Latino precincts (which numbered 11 nationwide in the 2004 exit poll) but not available in the thousands of other precincts they select. The result is that the total share of all Latino interviews conducted in Spanish is extremely small. In 2004, for example, 35 interviews were completed in Spanish out of 638 Latinos in the national NEP dataset, or 4.7%. In short, exit polls are under-surveying Spanish-dominant Latino citizens by a huge factor, introducing a systematic bias toward more assimilated, higher income, better educated voters. This is despite the fact that a 2011 national survey of Latino voters by Latino Decisions reported that 31% watch Spanish TV news programming every day, and an additional 25% watch Spanish news programming multiple times per week. Only 22% of Latino voters said they never watch Spanish TV. By offering a very limited amount of high-density Latino precincts, and very limited interviews with Spanish dominant Latinos the national exit polls dramatically misrepresent the Latino vote.
3) How reflective is the exit poll of the true population of Hispanic/Latino voters?
Exit poll demographics vary meaningfully—and consistently in the same direction—from Current Population Survey data regarding the composition and characteristics of the electorate. That is, the exit polls over-represent middle-class, higher educated minorities. This may be a result of the selection of precincts and it may be a consequence of the language problem. Results from 2008 and 2010 illustrate how significant the skew is.
Table 1 illustrates the demographic skew in the 2008 Exit Poll. For all non-white voters (which is how NEP reports its demographics), the share of voters with a college degree is approximately 12% higher than in the Current Population Survey’s November 2008 supplement. This represents an almost 50% increase in the share of minority voters with college educations in the exit polls vis-à-vis reality. Likewise, the share of minority voters making above the national median income is 5% higher than it should be, representing an 11% error. In both cases, the income and education levels are shifted toward higher status, less Democratic, minority voters. Table 2 illustrates the effects in the 2010 exit polls compared with the 2010 Current Population Survey. They are almost identical in size and effect.
The effect in individual states can sometimes be even greater. Since state samples are smaller, the potential for error is greater. Table 2 illustrates the deviation in educational attainment, by state, between the NEP and the Current Population Survey for 2010. Colorado is the most egregious case, where there is an almost 30 point gap between the share of exit poll respondents with a college degree and the Census Bureau’s estimation of the same. In Arizona the gap is 20%, in Florida around 13%. As is readily apparent, the minority voters who appear in the 2010 National Exit Polls are not representative of the overall distribution of minority voters who cast ballots in that election.
“ELECTION EVE” METHODOLOGY AND ACCURACY
In 2010, Latino Decisions conducted an “election eve” poll of extreme high propensity voters. Fielded on the nights just before the election, those polls interviewed only respondents with a high-attendance vote history who reported having already voted (early or absentee) or reported being “certain” to vote. We surveyed 3200 respondents in that effort, and we subsequently ‘validated’ their vote and determined that 88% of them actually voted in the 2010 election.
How accurate were our assessments? We collected precinct level voting returns from the 2010 election, as well as demographic data about the composition of the precincts. We used Goodman’s Ecological Regression (ER) and King’s Ecological Inference (EI) to estimate statistically the actual Latino vote share received by each candidate. While somewhat different, these two procedures will provide the best estimate of Latino two-party vote since it is driven by the actual votes cast and the actual demographic composition of each voting precinct. The results are presented in Table 3.
The results overwhelmingly suggest that the LD estimations of two-party vote share are far closer to whatever the underlying truth might be. The NEP estimates are off, way off. The NEP estimates differ from the estimates based on the actual votes cast by 11-24%! By contrast, LD’s estimates were consistently closer to the precinct estimates. In fact, in every race LD studied in the 2010 election, the mean absolute deviation of the precinct estimate from our poll was 3.25%, and the mean overall deviation was .34%. By contrast, across those same races, the NEP mean absolute deviation from the precinct ecological estimates was 17.06%. And the men overall deviation was -17.06%. These numbers are the same because all of NEP’s errors were in the same direction, systematically underestimating Democratic vote share.
STAY TUNED FOR 2012
Latino Decisions, in coordination with impreMedia, is conducting a 2012 Election Eve Poll. This poll will include 5600 Latinos, covering national estimates and 11 state stratified samples in California, Texas, Florida, Arizona, New Mexico, Nevada, Colorado, Virginia, Massachusetts, North Carolina and Ohio, as well as a national 50-state election eve poll of Latinos, to be released on election night, November 6th. Collectively, these polls will provide a more accurate and nuanced portrait of the Latino vote and its effects in the 2012 election.
