David C. Wilson
November 20, 2009
Who’s being blamed for the recession?
This is the first note in a series of briefs related to the presentation of political polling information from various outlets. The Toplines are the different numbers presented in a report or new story containing poll data, and the headlines are of course what is supposed to be the “best” summary of the information to follow. The goal of “Toplines and Headlines” is to help the average consumer of politics to better understand how polls are used to support narratives, or broader themes the authors, producers, or outlets are attempting to pose to the public. Readers should not consider these briefs a criticism of polling or poll data, but rather use these notes to question the inferences made by those providing the story.
Today (11/20/09) the CNN website released a headline that read, “Poll: Public shifting blame for recession.” The story used data from a November poll of U.S. adults (N=1,014) conducted between the 13th and 15th, with a “margin of error” at plus or minus 3%. The headline suggests that there is a meaningful change in who the public is blaming for the recession. The topline information found that in May, 53% of the public blamed the Republicans for the recession, and that number is now only 38%, a 15% decrease.
The CNN Polling Director, Keating Holland notes this is a bad sign for the Democrats because if the numbers continue to decline “only a handful of Americans will blame the economy on the Republicans” by the time the 2010 elections roll around. Yet, according to the poll, 27 percent now blame the Democrats for the recession, up 6 points from May (33%), and 27 percent now say both parties are responsible.
A sophisticated poll watcher would raise several concerns after reading this story. First, when using samples “point” estimates (a single number) are less important than “interval” estimates (point estimates with the margin of error considered). The margin of error for this poll, as well as the others, is at least 3%, which means the numbers for Democrats is anywhere from 24% to 30% now, and 30% to 36% back in May; thus, while the Republican blame numbers have clearly decreased, there is a chance that the numbers for Democrats haven’t changed at all: 30% then and 30% now.
Second, watch out for the shell game with numbers. More Americans still believe the Republicans are responsible for the recession (by 11% over Democrats), and while the Republicans have decreased public blame by 15% and the Democrats have only decreased by 6%; why is the shift not uniform for the parties? The real number to keep your eye on is the one that hardly gets any mention: the percent of those who blame “both parties” which is now at 27%, a number which is equal to the Democrats numbers. While the Republicans are not being blamed as much as they were 2 months after the President took office (i.e., the honeymoon period), the blame is not necessarily shifting “to” the Democrats. The group taking the main hit on the recession is actually Congressional incumbents for both parties. According to Gallup’s daily tracking, these numbers has been on a steady decline since March of 2009.
Third, beware of the opinions behind the inferences; evidence is more important than guessing. Keating suggests that growing concerns over the budget deficit may be to blame; stating that two-thirds believe the government should balance the budget even in times of war. However, there is no evidence in the story that the two beliefs—who’s responsible, and the budget deficit—are connected. It would be a simple task to “cross-tab” the two questions to show that those who believe the budget should be balanced are also more likely to blame the Democrats for the recession. The omission of this information should always raise concerns about the strength of the claims.
Fourth, keep in mind the context. Since May, the American public has seen the passing of an unpopular stimulus bill, the passage of the largest budget in the history of the country, a fierce ideological and partisan debate about health care, and a contentious set of November 2009 elections, and most poll results comparing Democrats and Republicans, as well as Obama to other groups, still show statistically significant advantages for the Democrats and the President. This is particularly clear, when the public are asked “who do you trust more to handle the country’s problems?” and the President has the highest numbers, followed by the Democrats, and then the Republicans.
In summary, it’s important to remember that headlines are a means to an end; they persuade us to read a story as well as establish our expectations for what’s to come. The challenge is that by anchoring our expectations they can often lead to us assimilating what’s in the story to the actually headline, rather than evaluating the merits of the points in the story. This is particularly dangerous with poll data, and its quantitative literacy requirement. Remember to always keep in mind that trends are not always comparable, that inferences based on opinions and not facts can lead to misreading the public (this happens a lot), that numbers by themselves are not a “story,” and always keep in mind the context when thinking about what the numbers actually indicate.