Surveys produce precisely inaccurate findings

In New Brighton, Minnesota, my old home town, the city council paid $4,600 for a a survey that asked how many residents voted in the last election. It found that 47 percent of the 400 respondents said they "always" vote when, in fact, less that 18 percent showed up for the last election. Professor Sandford Weisberg, director of the University of Minnesota's Statistical Consulting Service, wasn't surprised by this. He says that "people always want to say what pleases people." However, the pollster hired by New Brighton claims that the people he surveyed simply "misremembered" that they hadn't voted. A recent article in New York Times opinion pages* provides much more alarming evidence of misleading surveys, for example, one by American Medical Association (AMA) that reported an alarming rate of binge drinking and unprotected sex among college women during spring break. The AMA survey, supposedly based on a random sample of 644 women, provided a margin of error of +/– 4 percent. However, according to the Times, the survey included only women who volunteered to answer questions — and only a quarter of them had actually ever taken a spring break trip! The article goes on to cite other cases of surveys that produced very misleading results, including one similar to the one done by New Brighton. Beware of what you read about what other people think, especially if it comes from a scientific survey.

*Precisely False vs. Approximately Right: A Reader’s Guide to Polls, by Jack Rosenthal, August 27, 2006