By now most of you probably heard about the "disaster" at the Daily Kos. It turns out their polling company, Research 2000, has been faking their polling data. **snicker snicker** This has, of course, called into question the validity of all the “analysis” they’ve done over the past. . . well, forever. But their loss is our indifference because liberals believing wrong things is as normal a part of life as breathing. But that raises the question, how do we know what the public is really thinking?
The most obvious answer remains polls: polls can be accurate enough to give you a fair impression of the public’s mood. However, we must always remember that polls suffer from serious limitations. For example, polling relies on obtaining a representative sample. That means they need to poll a large enough group to be able to make certain statistical assumptions, and that group must match the broader population.
And that’s really where the first problems appear. How many Democrats should you include in a poll? How many Republicans? Can you be sure that your independents represent actual independents? It is the answers to these questions that cause poll results to vary so much, even if they all used the same questions. But that’s yet another problem: they don’t ask the same questions, and depending on how the questions are phrases, which words are used, and even the available answers, it is possible to manipulate a poll to such a degree that it no longer represents the public.
But there’s another aspect to polling that needs to be considered: the lack of commitment or intensity. And by this, I mean that people will say things but won’t really mean them. The best example of this comes from environmental polls. “Should we try to stop global warming?” A large percentage of people will answer “yes” to this. So we all believe in global warming and we support cap and trade right? Not quite. When they asked “are you willing to spend $100 a year to stop global warming,” the “yes” numbers crashed. By the same token, if you ask “which of the following candidates would you support in 2012?” and then you list ten candidates, it’s possible that all ten candidates will score 90% approval ratings. But does that actually tell us anything? It really doesn’t because there is no ability to distinguish the intensity or the level of commitment.
The same applies in other areas as well: “Are you so angry at Hollywood that you intend to boycott all future movies?” A sizeable percentage of people will answer “yes” to this. But there’s no way to tell if they are serious. In fact, this question is a very easy way to voice displeasure without undertaking the commitment of following through.
So how do we really know what the public is thinking? You follow behavior and you look for patterns in it. For example, the public says they love candidate X, but is anyone giving them time or money? Are they drawing a disproportionately larger share of money or volunteers than other campaigns? Do they draw larger crowds at events?
It’s the same thing with movies. You see a series of trade publications giving all kinds of awards to actor Y. He’s voted the sexiest man alive, and is cast in dozens of films. The headlines scream -- “everybody loves Y.” But do the box office figures concur? Whether or not people will spend money to see him is what counts. A recent example that demonstrates the importance of following the money is the movie Snakes on a Plane. This turdburger was hyped all over the internet and easily won the “most clicks” war for several weeks before the film came out. But clicks are free, admission isn’t. The movie tanked. Once again, there was a difference between what people said (clicks) and what they did (pay for a ticket).
When I write articles about the public accepting one thing or rejecting another, these are the things I look for. Did the movie make money? Can the movie star sell tickets? Does candidate X actually engender monetary support or just “clicks.” Does the public actually support the actual regulations that will affect them, or just the principle of a happier world?
You can even use anecdotal evidence if you are careful about it. What you need to look for are inconsistencies. For example, the fact that your neighbor complains about Obama means nothing. If they’re conservative, then this is to be expected. If they’re liberal, then it could just be blowing off steam. But a non-political neighbor who expresses anger at something they wouldn’t normally know anything about, can be informative. It can tell you, for example, that what Obama has done has drifted down into the public consciousness and not in a good way. That’s how it first became obvious that the BP spill was a problem for Obama -- people who don’t pay attention to politics were suddenly talking about his response.
Moreover, you can look at how the left reacts. For example, the more shrill the denials that a particular event has any meaning, the greater the likelihood that it actually has struck a chord and they are worried. This is what confirmed that the public was upset about the BP spill, because leftists were falling all over themselves to write articles telling us that no one cared.
This is also how I can tell you that financial reform means nothing to the public. First, note the lack of articles about financial reform. That’s because journalists aren’t going to write about something the public doesn’t care about. Secondly, note that when we write about it, the numbers of comments drop dramatically and the comments themselves become very short. This isn’t a lack of brainpower on the part of readers, it’s a lack of interest. Add in the falling ratings of the business news channels and the disappearance of business news from newspapers and nightly news shows, and what you get is the public voting with their feet, i.e. they are changing the channel because they don’t care.
Further, you can often derive fascinating information about the public from the strangest sources. Indeed, one of the most brilliant strategists the Republicans ever had, the late Lee Atwater, once said that he would read the National Enquirer every day. The reason was that this newspaper of made up news told him what the American public was thinking about.
Examples of the kinds of things I’m talking about here could include (1) the lack of success of energy efficient products, which indicates that consumers aren’t willing to substitute environmentalism for quality; (2) the lack of success of recent Megan Fox films, which could signal public anger at her insulting political statements; (3) surges in gun sales and security systems in certain areas, which indicate a loss of confidence in the local police; or (4) consumer spending going down even as consumer confidence polls are going up.
And you can also get to what the public really values in this way. For example, a few months back, there was an interesting tidbit from Netflix. People keep serious movies significantly longer than the mindless stuff. The reason, it turns out, is that people rent the serious stuff because “they should,” but then never want to watch it. In fact, many of these films are kept for several weeks before being returned unwatched. What does this tell us? It tells us that despite protestations that the public likes “deep” and “serious” entertainment, it actually prefers the mindless stuff. How is that useful knowledge? Well, it helps you understand the psyche of the public -- the public has an idea of what they should be like, but they don’t live up to that. So if you are planning a political campaign, this tells you not to make it hard on them. This also tells us to remember that the public may over-idealize itself in response to polls.
There are millions of more examples, but this should give you a sense of the kinds of places you can look to see what the public really believes. Polls are nice, but it’s actions that matter. And within actions, there are patterns and motivations.
To understand the public, consider the old saying turned around: “do as I do, not as I say.”
Thursday, July 1, 2010
How To Understand The Public
Labels:
Polls,
Psychology
Subscribe to:
Post Comments (Atom)
30 comments:
Andrew,
Great article.
I can see how it has gotten to the point where polls are driving stories and not the other way around.
Andrew,
The inference the statistic that mindless fare is turned over quickly means that we are mindless could be unfair.
Could it be that people who are in serious jobs, need mindless fare in the form of entertainment to balance out their lives? And serious movies don't provide that balance? And that reading too much into a statistic when you don't know exactly why they do what they do..... All the statistic said is that there is more turnover with mindless fare. Nothing could or should be inferred beyond that.
My point being that inferences about what is going on with the public can be skewed without that much effort.
For instance, there is a poll that Obamacare is gaining acceptance. I don't think so, because I have not changed my mind about it and haven't heard anyone who touts it except MSM and the government. Two sources extremely suspect for me. Could it be that the poll was worded a certain way and that an unfair inference was made in order to get the results that the meme demands?
Joel, Thanks! I thought it was a good thing to talk about, particularly in light of the Kos thing.
Too many people are letting polls do their thinking, without every doing any analysis of how reliable those polls are or even if they make sense. Sadly, journalists are trading traditional leg work, fact checking, and real analysis, i.e. thinking, for simply putting up opinions and finding polls that fit that.
I'll be right back on your second comment...
Attention span is a definite American flaw. Message by sound bite allows the MSM to distort the public information. One of the newest crutches for lazy “journalist” is making news with a poll. These polls have zero real value, but allows the scoundrels in the press to lead the zombies to a pre-determined narrative, and result. There are some good public polls out there, Rasmussen, Zogby, and sometimes Gallup, in that order.
The political insiders say the best polls, are internal polls. You can watch, and listen to the politicians, around election time, and they’ll tell you what they’re scared off, and what the internal polls are saying, if you pay attention.
Interesting stuff! I guess this is what lawyers do with their time off? ;)
On the film thing, it was combined with a survey by NetFlix that people were returning the videos unwatched, so it wasn't just that they held those long, but also that they weren't watching them. To me, that's a pretty good indication of an intent to see the "good" stuff, with a lack of follow through. You get similar data from responses to polls about eating healthy, giving to charity, etc. -- all the things "we're supposed to do."
From that, I infer that people are giving in to social pressure that tells them what they "should be doing", but that they lack the follow through. But that they also will report on polls and the such that they actually do follow through. So my point is that this data presents a disconnect between what we claim to believe and our behavior.
Could it mean something else? Sure, that's the difficulty with this kind of analysis. But if you look for patterns and verification in other areas, then you can put together a pretty good picture of humans.
As for the polls on ObamaCare, that's all about how they ask the question and the answers allowed. It also matters who they sample. For example, if they chose people who were "mildly opposed" before, they might now not care since it's no longer at the top of their minds. But those people probably aren't going to vote in November either. That's why I say you need to look at behavior. Have people stopped talking about it? No. Are the Republicans getting a bigger spine about repeals? Yes. Have the Dem's dropped their campaign that Republicans want to repeal it? Yes. That tells us that nothing has changed.
Stan, That's exactly true on both counts.
On the one hand, the politicians do extremely good polling (usually combined with focus group work) and they will let slip what is really going on with their actions -- look at how their ads change and the issues they talk about and those they don't.
And you're right about journalists using polls to make news. It's very easy to commission polls and then use those to tell everyone what they should believe because "everyone else believes it." In some cases, they even use these silly non-scientific polls as news. Those things are completely worthless.
By the way, I’ve been watching the DailyKos stuff and I think it’s hilarious. I read that Kos is trying to get the raw data from their polls but they won’t turn it over. At one point they even claimed their computers were down so they couldn’t turn anything over because they were working out of a Kinkos. I wouldn’t buy that excuse.
Crispy, According to Kos, they've been asking for the raw data and they've been getting all kinds of excuses. One of the excuses was that their computers were down that day and they had to work out of a Kinkos instead of their regular office.
What makes this even more interesting, is that Research 2000 had a lot of clients, including some campaigns. But the guy was a deadbeat -- there are several judgments against him from local stores including two grocery stories. BUT, Kos admits that he didn't pay his bill either.
So who knows what's really going on here? But whatever it is, it looks like two liberals are busy taking each other down! Can't say that bothers me.
here's my never-failed-me-yet method for understanding the public: i go to a grocery store, stand around the meat/produce/wine section and listen. sometimes i might say something like, "hey, how's it going?" and listen as the public rips. my unscientific polling tells me where john q stands, and right now, he's pissed at washington. overwhelmingly.
Andrew: In the spirit of schadenfreude, I enjoyed the Daily Kos "embarrassment" enough to expand on the subject in this afternoon's post. Your post does a much better job of looking at the significance of all polling (or insignificance, in some cases). Why people take polls as gospel when they're so often wildly divergent and just plain wrong is beyond me. But I think your comparison to NetFlix customers is probably right on the money. The most I can say of polls, is the better ones seem to be able to catch trends, and occasionally get very close to the final results.
Patti, That's a pretty good measure so long as you keep track of how the public mood changes. For example, around here (a conservative area) people are nearly in open revolt. That tells us something about voter intensity.
At the same time, in a liberal area where I've spent time, people aren't talking, and all the Obama bumper stickers have vanished -- which again tells us something about voter intensity.
These are all good signs you can read.
And when it gets to the point that people are talking about something in public at a meat counter, then you know it's struck a chord.
Lawhawk, Thanks. I agree about Kos. I can't wait to see what happens next, especially now that Kos has let slip that he hasn't paid the bill he owed to Research 2000.
How funny would this be if the whole problem was that Kos is going broke?
I also can't wait to hear what Research 2000's other clients do now. There are several newspapers and several campaigns that hired Research 2000, and they need to be thinking about taking action as well.
Not that it matters because Ali apparently has no money -- there are several judgments against him from local grocery stores (Giant and Shoppers -- good donuts).
In any event, it's going to be a lot of fun to watch this explode and get bigger and bigger. Especially since Kos has been the driving force on the left for some time now. . . and everything they said was garbage! :-)
Lawhawk, I also am still getting a real kick out of what Bev said yesterday about the Huffpo people being beside themselves over this. . . though I don't have the stomach to go check it out. I'm glad we have Bev to report though!
Andrew: I can't help but see a tiny comparison with the Afghan warlords. They often appear to be on the same side, but they'll stab each other in the back at the first opportunity. Just like the Huff Po and the Daily Kos.
I can't help but laugh at the fact that one leftist hero hasn't paid his bills and the other leftist is a fraud, and now they're going to war. . . like all leftists always do with each other.
It's like 1939 all over again, only on a smaller scale! :-)
andrew: i agree, when folks offer their opinions openly to total strangers, something serious is afoot. my libbie friend experience is that those who voted for this disaster are now silent and some have taken to outright hiding. it speaks volumes. oh, their shame.
Patti, It absolutely does.
Also, what you mention today at your website is another good example of what's really going on -- with the Republicans standing up for an actual repeal of ObamaCare.
That tells us that they know the public is ready for a repeal. Whether this knowledge comes from internal polling or volume of mail or simply the overwhelming town hall experience, that we don't know. But we do know that the stridency of their position tells us that they believe the public is with them.
Add in all the other evidence, and that becomes a pretty good bet.
It is my pleasure to sacrifice my mental health for "the cause" by keeping tabs at HuffPo.
Great article - The Clintons were masters at internal polling and trial balloons. The first "poll as news" I remember was the now famous Time Magazine poll question in 2003 or 2004 "Do you think the country is going in the right direction" -
85% of respondents said "NO".
Time Mag concluded that no one liked the direction to mean that BUSH was taking the country. This number was used over and over by every news organization as an indictment of the Bush adminstration.
I was surprised by that because I actually took the poll and voted "NO", but I interpreted "right direction" to mean "socially in the right direction", not politically. I don't think I am the only one.
Time got the outcome they wanted to get (85% against), but the poll was so broad that it could be and was interpreted to mean whatever the poll taker wanted it to mean. All it meant was that there were 85% that were disgruntled about something...
Thanks Bev! I'm glad you liked it.
And thanks for sacrificing your mental health! :-)
What you mention is another problem with polling. What does the 85% really mean? Do they want a shift to the right? To the left? Are they thinking of something different, i.e. not-political, like you were?
Those are the hazards of polling and, more importantly, the hazards of believing in polls.
For example, "do you oppose ObamaCare" is the same thing. Different people will oppose it for different and often contradictory reasons. So a generic number without more doesn't tell us anything. Similarly, to get real meaning, you need to ask "how does this affect your vote" and even then it's very easy to say "I'll never vote for them again" without really following through.
That's why it's so much more informative to watch how people actually respond to events than it is to listen to what they say.
That's also why Kos was stupid to rely so heavily on their unique polls as the basis for their analysis. As a supplement, sure, it's useful. But to rely as heavily as they do is simply poor reasoning.
Great article. I see all these things, but never put them together. From now on, I'm going to start looking a little closer! I swear I learn something here every day!
I think I have long realized a couple of the most salient points from your excellent post. 1) Polling is used as a political weapon. The so called "push poll" is particularly favored by the main stream media. They basically have an agenda or story to tell and want to report on those things that support the story. Sometimes, even they realize it is too hard to fake and to do so would harm their veneer of so called "impartiality." Thus, if a Republican is in office and something bad happens, the polls are out almost daily. When it happens to a Democrat . . voila; less polls.
The second point you mentioned is how people say what they think they "should" say rather than what they think. Just as with net flicks, this is sort of the derivation of the term "guilty pleasure." But it is why we worry about who is winning or losing the media wars. Many educated people have mixed feelings about letting everyone vote, myself included. We guage people below 18 as not mature enough to vote. We consider driving a priviledge, not a right. And yet, we require nothing of our citizenry to vote other than to register and even that is too much for many of our citizens. Fortunately, most of them don't usually vote, but we worry about those who can be swayed by bullshit polls and speeches.
Hi,
I just came across your site last week and have been really enjoying it. I'm definitely going to be more careful about what poll numbers I trust in the future.
D
Ed, I'm glad it gave you something to think about! That's our goal here. Hopefully, this will help you see things in new ways.
Jed, Absolutely. Polls have become a tool for pushing a narrative. In fact, I was reading earlier how the Kos polls actually changed the flow of money and volunteers in California and Arkansas. So these things aren't harmless, and people know this -- so they use them. And journalists have definitely learned to use these to push an agenda.
You're also right that people respond to polls with what they think people want to hear, rather than what they really believe. And the evidence for that is all around us.
And I think it's very insightful to connect that with voters being swayed by the wrong things, and with the liberal control of the media. As long as the media defines how you "should" act, then people will continue to report those kinds of opinions, even if they don't actually follow through with them. That leads to bad policy and the appearance of support for things that shouldn't be supported. And sadly, peer pressure is a powerful force with people. So if everyone says X, even if they don't believe it, then X can take on a life of it's own just based on peer pressure.
DUQ, Welcome and thanks for commenting! Hopefully the article has helped everyone see new ways to understand the public and given them a reason to be leery of polls that seem to go against common sense and what they observe throughout the country.
Great article! Thanks Andrew. By the way, I can't sign in so I can't get my picture to show. I don't know why.
Thanks Doc. I can't explain the problems with Blogger, but I do know that some people do have problems. Some people can't sign in, others can't stay signed in. It's not a great system.
We also have comments disappear for a while -- though they've always come back. And strange things will happen that we can't explain with posts at times.
Cool article. I think you're right about looking for what people really think in how they behave. There was an interesting article on Yahoo today about how subconscious ideas actually change out behavior without us knowing it.
Mega, I saw that article. It was pretty interesting. I'm thinking about writing a post about it. There has been a lot of evidence in the past decade that much of what we think we think is actually a response to subconscious stimuli.
What was most interesting was the part where they discussed how subliminal advertising doesn't really work, but what you see can actually be much more dangerous because it's triggering these subconscious impulses. Interesting times huh?
Post a Comment