Even if a polling organization is trying to do competent work, there are a lot of things that can go wrong with public opinion research.
So why do the media add phony polls to the mix?
When media sites ask members of the public to respond to a question they pose on-line, someone running the operation knows that the result is meaningless when it comes to telling you what the public thinks.
After all, you may even see a line noting that the results are “not scientific.” The Bangor Daily News does this when it prints the results in the newspaper (but it’s not on-line, at least not today).
If you don’t have a random sample — which doesn’t exist when people can self-select into a poll or survey — the results tell you nothing about public opinion.
Yet here’s the homepage of today’s Bangor Daily News with this:
Clicking on the link takes the reader to the “poll,” which looks like this:
And the results currently look like this:
Typically results from self-selected groups do not correspond with what real polls, using proper sampling techniques, find.
Just compare the above question to results from a recent CNN poll.
Now, the questions CNN asked are not identical to the one posted by the BDN and, admittedly, one would expect differences due to the divergent question wording.
For one, the BDN refers to “the Justice Department,” without tying the agency to a particular issue. Someone responding might be thinking of the investigation of the leaks by the Associated Press or they might thinking of something else entirely.
But both BDN and CNN asked about Obama’s relationship to and responsibility for the same issues.
CNN found that, on Benghazi and the IRS, conservatives and Republicans are harshly critical of Obama and do not trust him, but others think the president has been honest.
As Greg Sargent breaks it down:
* The IRS scandal: Among overall Americans, 61 percent say what Obama has said about the matter is mostly or completely true, versus only 35 percent who say it’s mostly or completely false. Among Republicans, 68 percent say what Obama has claimed is false, and among conservatives, 56 percent say this.But independents believe what Obama has said is true by 58-36, and moderates believe this by 71-25.
Meanwhile, among overall Americans, 55 percent say the IRS acted on its own in targeting conservative groups, while only 37 percent say the White House ordered it. Among Republicans, 62 percent say the White House ordered it, and among conservatives, 54 percent believe this.But independents believe the IRS acted on its own by 53-36, and moderates believe this by 65-29.
* The Benghazi story: Among overall Americans, 50 percent believe early statements about the attacks by Obama officials reflected what the administration believed at the time, while 44 percent believe they intentionally misled. Among Republicans, 76 percent believe they intentionally misled, and among conservatives, 65 percent believe this. But moderates believe the statements reflected the administration’s beliefs by 60-35. (Among independents this isn’t as pronounced, but still, a plurality sides with the White House, 47-44.)
Overall, a majority in the CNN poll thinks these are serious issues, but most (53%) approve of Obama’s job as president. Moreover, CNN reports that, “Only 43% say they have a great deal or some confidence in the people who run the federal government. But 56% say they have a great deal or some confidence in the system of government.”
Again, admittedly, these are all different questions. As such, they will pick up different elements of public views. If these were both polls done with proper sampling, that could lead to a nuanced analysis of how people respond to varied questions on the same topic.
But the real problem with the media on-line question is the lack of a random sample. People self-selected into the response group.
So different results from CNN and the BDN aren’t surprising
Question wording aside, consider this: Typically on-line questions spark interest from people with the strongest views.
And, as Gallup showed the other day, the most engaged on these issues are conservatives and Republicans.
54% of Americans are following the IRS investigation closely, but that’s true for 67% of Republicans. The numbers for Benghazi are similar.
Thus one could expect Republicans and conservatives to respond to this question disproportionately to their presence in the population. Other questions on different issues would include disproportionately more of other groups.
My assumption is that media outlets run the phony polls because they may get clicks, get shared in social media, and generate more readership.
But are they news?
Decades ago, journalist and media critic Walter Lippmann defined news as, “A picture of reality on which citizens can act.”
Phony polls don’t give you a picture of reality, so they’re not news.
And no one in the news business should have anything to do with them.