How to Lie with Statistics

Content Warning: The source material and this guide include discussions of suicide and systemic racism.

Chapter 1 covers the problem of sample bias in statistical data. Darrell Huff explains that it isn’t possible to count every individual when studying a population, so a statistician needs to create a sample representing the whole. To be the most accurate, a representative sample needs to be “one from which every source of bias has been removed” (20). Sample bias, he notes, appears in many different forms.

Huff introduces the chapter’s issue with an example: the reported amount of money made by an average graduate from a given year. The problem with the statistic, Huff says, comes from biased sampling. All the graduates were probably not contacted, and those who responded were likely more influential and made more money than those who chose not to. That creates a bias in the sample toward one part of the population over the other and yields an inaccurate result.

The following section covers the problem that the reported numbers could be made up, even with a decent sample. Huff’s examples here highlight that people often lie to make themselves look better in polls, from how much money they make to the media they consume or even their health habits.

Huff points out that invisible biases cause as many problems in statistics as visible ones. Aspects of a population that might lead to part of it being overlooked or overrepresented are the product of the biases of the person taking the measurement. In Huff’s examples, these often come from a lack of access. For instance, he cites a 1936 presidential election telephone poll of 10 million Literary Digest subscribers that is often cited as a statistical disaster for predicting that Alf Landon would defeat Franklin Delano Roosevelt. The results were biased because the sample failed to account for people who could not afford a subscription and a telephone in that era, so they skewed heavily toward Republican voters. The bias of the people taking the polls can also cause issues; Huff notes that the race of the person doing the interview and of the person interviewed can create different results due to tensions that result from systemic racism.

Huff concludes the chapter by discussing random sampling. While truly random samples are ideal, the cost and difficulty of getting them requires using stratified random samples instead. Huff says biases can never be removed entirely from a sample, and the reader should know where bias could appear in any statistics they encounter. In his examples, he references the work of Dr. Alfred Kinsey on women and sexuality. He notes that Kinsey’s samples, while not random, are large enough for significance and should be respected, but too much information should not be assumed based on them.

His final note for the chapter is that polls are often not rigged intentionally. Instead, they tend to bias toward those who are more “appealing” to those creating the poll. This usually means favoring people who are educated, wealthy, and able-bodied.